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Abstract 

‘Modem Times’ bring change in all areas of life. Even the rate of change has 
changed and is still changing to increase its enormous speed even more. Newer and more 
challenging questions face the military analysts everyday putting a question mark at the 
end of their last answer. Recently, the question of how to structure a robust air force to 
meet the requirements of competing, uncertain future scenarios has been keeping analysts 
busy. The new world order does not tolerate only being able to respond to a single 
scenario anymore, which once was considered a hard problem. Who knows what comes 
next? 

In this thesis, we employ a robust optimization methodology to provide an answer 
to the multi-scenario optimization problem. The methodology uses a meta-heuristic. 
Scatter Search, to guide the search of the multi-scenario solution space obtained by the 
evaluations of Combat Forces Assessment Model, the model currently used to respond to 
single theater scenario objectives. A Visual Basic\DOS routine performs the necessary 
interactions to find an Air Expeditionary Force structure robust across three notional 
threat scenarios. 
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ROBUST MULTI-SCENARIO OPTIMIZATION 


OF AN AIR EXPEDITIONARY FORCE 
FORCE STRUCTURE APPLYING SCATTER SEARCH 
TO THE COMBAT FORCES ASSESSMENT MODEL 


I. Introduction 


Thesis Overview 

This chapter starts with a brief description of the multi-scenario optimization 
problem including its background, and continues with brief discussions of the research 
objectives, approach methodology, and scope of the thesis. Chapter Two provides 
information from the current literature about the tools and methodologies that address the 
multi-scenario problem while Chapter Three discusses our research methodology. 

Chapter Four presents a comparison of our test results to those of a previous study in this 
area. The final chapter discusses the research findings and proposes recommendations 
for further research. 

Background 

Today’s amazingly fast changing world presents the military with the challenge of 
being ready for an unexpected crisis situation anytime, anywhere since this unexpected 
situation would not just affect the countries involved. Such situations would have global 
consequences. Thus, many military planners have been trying to answer the main 
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question of this readiness problem: How to build a force structure with the necessary 
equipment to best meet all of the competing requirements of different, uncertain, future 
scenarios. Clearly, to have a huge force to guarantee the best outcome in all potential 
conflicts is not an option due to the tremendous cost. The answer lies in finding a way to 
size a robust force structure across the range of uncertainty represented by the scenarios. 
The term robust refers to being stable under different conditions, or not showing a 
difference in the output when the inputs are changed slightly. So, we need to find that 
force structure which meets the objectives of all the scenarios. 

One proposed solution pertains to the US Air Force’s restructuring into an 
expeditionary aerospace force (EAF) with ten aerospace expeditionary force (AEF) units 
containing 120-150 aircraft. The foundational question for this research is how to 
determine the aircraft and weapons mix that are best for an AEF. 

Problem Statement 

To meet the Air Force’s need to be prepared for any situation anytime, analysts 
need a methodology to determine force structures that are robust across uncertain future 
scenarios. Specifically, for our test case, the number and type of aircraft and munitions 
needed to form the force structure must be determined. The problem is how to size a 
suitable force to meet the uncertain demands of future scenarios. We propose a 
methodology for finding robust solutions to military force planning problems. Our focus 
will be on building an air expeditionary force structure that is robust across three notional 
scenarios. The research using the existing Air Force Studies and Analyses Agency’s 
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(AFSAA) model CFAM (Combat Forces Assessment Model) to evaluate the force 
structure. 

Research Objective 

The objectives of this thesis are to develop and apply a methodology for 
structuring an air expeditionary force robust across a variety of scenarios, compare the 
results with previous work accomplished in the same area; and identify alternative ways 
to better examine the solution space before coming to any firm conclusions. In addition, 
the methodology is demonstrated in a way applicable to realistic large-scale problems of 
the same type. Thus, our methodology may aid in planning and budgetary exercises. 

Approach Methodology 

This thesis employs Scatter Search, an advanced heuristic search procedure, to 
guide the use of the Conventional Forces Assessment Model to evaluate the effectiveness 
of candidate force structures across the multi-scenario space. The methodology 
terminates with robust solutions. The basic research approach is diagramed in Figure 1. 



After Certain Number of Iterations 


Figure 1. Basic Research Approach 
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The interface performs necessary interactions between the meta-heuristic and the 
combat model. First, it runs CFAM with a candidate solution suggested by Scatter 
Search. For each scenario run in CFAM, outputs are transformed into a scenario 
evaluation. After performing this evaluation three times (one for each scenario), the 
interface combines the evaluations into a single weighted objective function value, which 
it passes back to the meta-heuristic to continue the search of the unexplored solution 
space. The cycle stops after a specified number of iterations or when convergence to a 
global optimum is detected. 

Scope 

This thesis considers an AEF sized notional force structure tasked against three 
potential, seven-day duration, small-scale contingency (SSC) scenarios for the 
comparison of two heuristic search methods, the Scatter Search and Bennett’s GA 
(Bennett, 2000) applied to a multi-scenario optimization problem. Therefore, Bennett’s 
settings are used. Bennett selected an AEF structure because of the U. S. Air Force EAF 
emphasis and considered the SSC instead of a MTW (major theater war) due to its 
increased likelihood of occurrence (Bennett, 2000:9). To measure the effectiveness of 
candidate force structures, the combat model currently in use by the US Air Force, 
CFAM, is used as the evaluator. The problem is chosen to be relatively small because of 
the long processing time of CFAM runs, assuming the methodology can be scaled up to 
meet the requirements of real life scenarios. 
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II. Literature Review 


Introduction 

To address the multi-scenario optimization problem, solution techniques available 
in the operations research literature are employed. Some of these are covered in this 
section. Beside heuristics, there also are other optimization techniques such as 
exploratory analysis, robust optimization, scenario analysis, stochastic modeling, and 
graceful degradation. 

Heuristic Search Methods 

A heuristic search method is a technique used to solve optimization problems, 
which seeks good (i.e. near optimal) solutions at a reasonable computational cost. 
According to Nicholson, it is a procedure "... for solving problems by an intuitive 
approach in which the structure of the problem can be interpreted and exploited 
intelligently to obtain a reasonable solution” (Nicholson, 1971:111). According to 
Muller-Merbach, the term heuristics is usually understood in the sense of an iterative 
algorithm, which does not converge toward the solution of a problem (Muller-Merbach, 
1981:1-23). 

A meta-heuristic is used to define master strategies modifying other search 
procedures to find solutions other than those that are produced while searching for local 
optima. To answer the question why take a meta-heuristics approach in solving multi- 
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scenario optimization problems, some additional terms in regards to complexity theory 
need to be defined. 

A problem is said to be solvable in polynomial time if its solution algorithm has a 
worst case running time with 0(n k ), i.e. an asymptotic upper bound of (n k ) where n is the 
problem size and k is some constant. These problems are considered easy to solve as their 
solution times are considered reasonable. However, many practical optimization 
problems do not have solution algorithms guaranteed to complete in polynomial time. For 
these harder problems, other solution techniques are required. 

Heuristics are powerful alternatives when an analytic solution to a problem does 
not exist or is computationally too expensive to find. In addition, they are often easy to 
implement. Although heuristics do not typically give an optimal solution, they locate 
reasonably good solutions very fast. They also deal with the real problem versus an 
approximation of reality with strong assumptions. The following discusses examples of 
heuristic search methods: 

Genetic Algorithms (GAs) 

“Genetic algorithms are a new artificial intelligence paradigm that arose from the 
investigations of cellular automata in 1970s” (Buckles and others, 1990:28). The formal 
theory was developed by John Holland and in recent years has been applied to problems 
as diverse as pattern recognition and optimization. According to Webster’s dictionary 
(Kryger and Bubriski, 1999:485), the word genetic means “of or relating to or involving 
genetics,” where genetics is a branch of biology that deals with the heredity and variation 
of organisms. 
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Genetic algorithms (GAs) function by applying an analogy of genetics. A GA is a 
population based heuristic where good solutions, or chromosomes, survive into a new 
population to form a better generation in terms of overall quality of solutions. The set of 
solutions to a problem, making up a population, are also called vectors, or strings that 
consist of variables, or genes. Applying GAs to rule-based classifier systems, Holland 
gives an example of a basic execution cycle (the ‘central loop’) as follows: 

1. Select pairs from the set of classifiers according to strength-the stronger the 

classifier, the more likely its selection. 

2. Apply genetic operators to the pairs, creating ‘offspring.’ Chief among the 
genetic operators is crossover, which simply exchanges a randomly selected 
segment between the pairs. 

3. Replace the weakest classifiers with the strongest offspring. 

The effect of this procedure is to emphasize various combinations of defining 
elements -schemata- as building blocks for the construction of new, better solutions 
(Holland, 1986:42). 

The solutions are encoded, usually in binary coding, which most resembles 
genetic DNA structure. The genotype is the structure of the solution used by the GA 
while phenotype is the physical expression of that structure. For example, the phenotype 
of the number 8 has a genotype of (1000) in 4-bit solution space while the phenotype of 7 
has a genotype of (0111). 

A GA starts with an initial population. We can create an initial population by 
randomly selecting the 0-1 values of genes in each solution. Another method is to use 
another heuristic to obtain better than random members. However, this approach 
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increases the possibility of premature convergence, i.e. we end up with a local optimum 
early in the procedure. 

Selecting parents for mating requires a choice rule. The fitness value of an 
individual can be divided by the total population fitness value and used as a probability of 
parent selection. These approaches use fitness values. A specific proportional selection 
method, roulette wheel selection, assigns each individual a probability proportional to its 
fitness value. In rank selection, chromosomes are indexed by increasing fitness value 
with selection based on the index. The chromosome with the greatest index has the 
highest probability of selection for reproduction or mating. According to Grefenstette and 
Baker, the primary advantage of ranking over proportional selection is that the resulting 
algorithm is less prone to premature convergence caused by individuals that are far above 
average. (And, one might argue, selection by ranking seems closer to the mechanism of 
natural selection). A number of other alternative selection algorithms have also been 
studied by researchers such as Whitley, Goldberg, and Baker, but have not enjoyed the 
same level of theoretical analysis as the proportional selection algorithm (Grefenstette, 
1999:127). One of them, tournament selection, is a method where the best of randomly 
selected parents are retained for production. The common element of these selection 
methods is a focus on providing better solutions a greater chance of survival, either 
directly or via a mating process yielding offspring containing characteristics of the ‘good’ 
parents. 

Once parents are selected, offspring are generated. Offspring are generated by 
crossover and mutation operators. Both operators have a possibility of occurring. The 
crossover operator exchanges some parts of parent strings determined by a probability. 
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For example, suppose we have a randomly selected crossover point of gene location 3 
with Pi=(1001011) and P2=(0101010) as selected parents. Crossover, yields the offspring 
(1001010) and (0101011). This operator provides a means of exploitation, i.e. makes the 
algorithm intensity on a certain region of solution space. This is because more fit 
chromosomes will survive and get selected for mating. Then, the crossover operator will 
generate new solutions from these fit chromosomes giving the offspring the 
characteristics of the good parents. Here, higher fitness values may be found, but there is 
a high probability of getting stuck with local optima. 

Crossover serves two complementary search functions. First, it helps the search 
examine new points within the hyperplanes already represented in the population. 
Second, it introduces points from new hyperplanes into the population. By evaluating a 
string of length n, we gather information about the fitness of the 2 n hyperplanes 
represented by that string. GAs are so powerful mainly because they exploit efficiently 
this vast amount of accumulating knowledge by means of relatively simple selection 
mechanisms (Grefenstette, 1986:122-128). 

The mutation operator changes a randomly selected gene of a chromosome. For 
instance, a mutation at gene location 3 would produce (1011010) from (1001010). This 
operator gives rise to exploration, i. e. moves the search to other regions of the search 
space. 

A new generation is formed by some combination of parents and offspring. Each 
new generation can replace the old generation en bloc, called generational replacement, 
or incrementally, where members are killed, or removed, according to their fitness value 
(usually the objective function value). 
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The fundamental theory of GA is the schema (plural: schemata) theorem. A 
schema is a general set of chromosomes having some genes in common. For example, the 
chromosomes 1111 and 1001 are instances of the schema 1**1, where * is 0 or 1. This 
schema has a length of three (the distance between first and last non-* genes) and an 
order of two (number of non-* genes). 

We know that the most fit chromosomes are selected for mating and are carried 
over to the next generations. Each time we evaluate the fitness of a given chromosome, 
we are gathering information about the average fitness of each of the schemata of which 
that chromosome is an instance. We implicitly make use of this information to search for 
a best-fit schema, which has certain inherent solution quality attributes that survive 
generation replacements. 

GAs prefer short, low order schema. The distance between short and low-order 
schemata’s first and last non-* genes is small making it less likely to get separated by a 
crossover operator. Being less likely to break, short and low-order schemata will be 
carried over to the later generations (This assumption is called ‘the building block 
hypothesis’). Thus, these schemata are preferred since they have a high probability of 
survival. Holland comments “Genetic algorithms, using the strength as ‘fitnesses,’ offer 
subtle ways of discovering good building blocks, and there are new versions of theorems 
from mathematical genetics that enable us to understand this discovery process” 

(Holland, 1986:101). 

The genetic algorithm continues for a specified number of generations or until 
population convergence is detected. Beasley and Martin define convergence as, “A gene 
is said to have converged when 95% of the population share the same value. The 


10 




population is said to have converged when all of the genes have converged” (Beasley and 
Martin, 1993:58-69). They also claim that a properly designed GA will converge to the 
global optimum, by the principle of survival of the fittest. Even simple GAs have been 
reported to find good solutions faster than most other general solution techniques. The 
application of survival of the fittest means, only good solutions are kept and by searching 
a sufficient amount of the solution space, we will end up with a very good solution, if not 
the global optimum. 

Empirical studies show that some important parameters in designing a GA are: 
crossover and mutation rates, population size, population initialization, and parent 
selection. High rates (0.8, 0.9) for crossover and low rates for mutation (0.02,0.03) are 
found to yield good solutions. In addition, seeding the population with solutions provided 
by other heuristics yield better results than random initialization (Grefenstette, 1986:122- 
128). 

For population size, 30 is recommended as well as a value between n and 2n by 
Alander, where n is the number of variables (Alander, 1993:65-70). However, Holland’s 
n 3 estimate is the most frequently cited theoretical development related to population 
size. Goldberg states “Simply stated, this computation says that the number of schemata 
processed effectively (those processed with an error rate less than a specified value) is 
proportional to the cube of the population size- is n 3 ”(Goldberg, 1989:117). His research 
suggests that relatively small populations are appropriate for serial implementations and 
large populations are appropriate for parallel GAs. 

GAs are very domain independent, i.e. many problems can be encoded and solved 
by computerized GAs regardless of the original problem. De Jong reports, “Much of the 
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interest in genetic algorithms is due to the fact that they provide a set of efficient domain- 
independent search heuristics... There is now considerable evidence that genetic 
algorithms are useful for global function optimization and NP-hard problems” (De Jong, 
1988:121). 

In addition, GAs are robust due to the absence of any linearity-type assumptions.. 
As Holland points out “Moreover, because a genetic algorithm uses a distributed database 
(the population) to generate new samples, it is all but immune to some of the difficulties- 
false peaks, discontinuities, high-dimensionality, and so on- that commonly attend 
complex problems” (Holland, 1986:105). Furthermore, GAs are amenable to parallel 
implementation to solve problems more quickly. 

On the other hand, GAs are criticized due to their problem-independent nature, 
since problem-specific information can enhance heuristic performance. Another 
drawback is that GAs do not use memory for responsive exploration. Therefore, GAs 
may not be seen as clever search procedures. Finally, De Jong points out an important 
issue in his machine learning research paper: “We have seen that in most cases 500-1000 
samples must be taken from the search space before high-quality solutions are found. 
Clearly, there are many domains in which such a large number of samples is out of the 
question”. He also discusses that the difficulty of choosing a good internal representation 
for the search space increases with the complexity of the search space. Similarly, one 
must be careful in providing an effective feedback mechanism (De Jong, 1988:124). 

Scatter Search and Path Relinking 

Scatter search, uses combinations of reference points to examine a solution space. 
Tabu search employs path relinking to move between reference solution points. Not 
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surprisingly, these two heuristic methods are often viewed and presented as a pair 
(Glover, 2000). These are also methods implemented in OptQuest. 

Reeves and Yamada incorporated path relinking in their flowshop sequencing 
application. Consider two locally optimal solutions to a combinatorial problem such as 
the n-job, m-machine permutation flowshop sequencing problem. They claim “If the 
operators we are using induce a big-valley structure, then it is a reasonable hypothesis 
that a local search that traces a path from one solution to another one will find another, 
because local optima tend to be found near other local optima” (Reeves and Yamada, 
1998:45-61). They further say that even if no better solution is found on such a path, at 
least some knowledge is gained about the relative size of the basins of attraction to these 
local optima along one dimension. 

Reeves and Yamada developed a specific neighborhood strategy to select certain 
solutions from a variety of alternatives. Making use of the idea of ‘optima linking’ in the 
context of a binary space, a certain path was taken. Among all those that would take the 
current point one step nearer to the target solution, this was the one traced by finding the 
best move at each step. 

By parameterizing problem data and then generating solutions based on 
manipulation of the parameters, scatter search applies to many different problems such as 
neural network training, graph drawing, unconstrained optimization, vehicle routing, 
multi-objective assignment, and mixed integer programming. As it is in every search 
procedure, to begin with, we need to create an initial subset of starting solutions. 
Naturally, better starting solutions will serve in getting better results. To find better 
starting solutions, we can generate a starting set of solution vectors by heuristic processes 
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designed for the problem considered (Glover and others, 1999:299).After generating the 
initial solution, a subset of best vectors is created and labeled as reference solutions. 

Next, new solutions, which are linear combinations of the reference solutions, are 
generated. Finally, a subset of the best vectors is used as starting points and the procedure 
is repeated until a specified number of iterations have been completed. 

There are similarities between scatter search and the GA formulation of Holland. 
Both are population-based approaches, starting with some collection of solutions that 
evolve according to some guiding process. 

However, a major difference between the heuristics lies in combining solutions. 
GA’s crossover is very different than the rounded linear combinations of scatter search, 
which will be explained next. Furthermore, parent selection also has different 
implications. Early GAs took the approach of complete randomization whereas scatter 
search always focuses on good, elite solutions (Glover and Kelly, 1995:111-134). 

Glover and Kelly claim linear combinations provide a more varied set of 
possibilities for creating new solutions than GA crossover. They say that linear 
combinations also avoid the artificiality of resorting to the binary representations as GAs 
do. To illustrate this point, let us create integer solutions that are combinations of the two 
solutions x = 8 and x = 14. Every integer point from minus to plus infinity on the line 
joining x = 8 and x = 14 can be generated by rounded linear combinations. However, if 
we utilize binary representations of length 4, (1 0 0 0) for x = 8, and (1110) for x = 14, 
then the possible outcomes are more limited. In particular, linear combinations of the 
binary vectors (1 0 0 0) and (1110) yield the schema (1 * * 0) where the elements’ 
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can be 0 or 1. Hence instead of producing all possible integer points, these combinations 
produce only the two vectors (10 10) and (110 0), corresponding to x = 10 and x = 12. 

Path relinking adds features to scatter search. Linear combinations of the best 
solutions are regarded as paths leading to new solutions. The starting point of the path is 
the initiating solution and the endpoint is called the guiding solution. Both of the 
solutions are known as reference solutions. Linear combinations extending beyond the 
convex region defined by the reference solutions create an extrapolated relinking 
approach. 

To serve generating better reference solutions, multiparent paths are used. They 
are created by combining attributes from the set of guiding solutions. In this method, 
weighted attributes are used to select the better move. The generation of such paths 
relinks previous solutions that were visited before, in newer and more direct ways. These 
paths can also be traversed from both start and endpoints simultaneously searching two 
neighborhoods which means choosing among more alternatives. This approach is called 
variation in path relinking. The assumption of variation is that both small paths will 
merge at some midpoint. To avoid the danger of bypassing some solutions, crossing the 
boundary of feasibility is also made possible in path relinking. Tunneling allows the 
search to traverse an infeasible region in order to return to a feasible region. Strategic 
Oscillation, on the other hand, provides a more general means to cross boundaries 
between feasibility and infeasibility to find critical regions. If these oscillations to recover 
feasibility are deep, a means of diversification is achieved. Otherwise, intensification is 
performed to find an optimal solution. Another type of diversification is gained by using 
constructive neighborhoods for creating new solutions as in the vocabulary building 
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approach. Searching in a constructive neighborhood, incomplete or null solutions 
converge to the guiding solutions. Vocabulary building combines parts of elite solutions 
on the path to the endpoint assuming these elite parts will generate good solutions 
(Glover and others, 1999:297-315). 

Tabu Search (TS) 

Tabu search is an intelligent search procedure that uses adaptive memory to 
responsively explore the solution space of an optimization problem. Glover and Laguna 
state (Glover and Laguna, 1998:3): “A good analogy is mountain climbing, where the 
climber must selectively remember key elements of the path traveled (using adaptive 
memory) and must be able to make strategic choices along the way (using responsive 
exploration)”. A broader and more technical definition is: “Tabu search is a meta¬ 
heuristic that guides a local heuristic search procedure to explore the solution space 
beyond local optimality” (Glover and Laguna, 1998:3). 

To find a global optimum to a given problem, TS examines possible solutions, or 
moves, which consist of defining elements called attributes. Example attributes are arcs 
of a network problem that can be added or dropped to generate moves. Attributes are 
useful in the neighborhood search of given solutions as mentioned above. In addition, 
they serve to store solutions efficiently in memory. Rather than storing complete 
solutions as in the example of explicit memory structure, some distinctive attributes of the 
solutions are stored {attributive memory structure ). One of the features of explicit 
memory is that it tracks complete good solutions encountered. These solutions are used to 
expand the search. In cases where memory space is not a big problem, explicit memory 
can be used to prevent revisiting previous solutions. 
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On the other hand, attributive memory stores only the attributes gained from 
complete solutions via hash functions. These functions are used in cycle prevention by 
means of converting complete solutions to integer values, inexpensively. They are also 
used in storing complete solutions. If the same hash function values are generated for two 
different solutions, a collision is said to occur. To overcome this problem, a two-level 
method is used, where colliding solutions are converted to integers one more time. 

From a different classification point of view, short- and long-term memory types 
are also used to store the tabu-status of attributes beside the types of memories mentioned 
above. The status of an attribute is tabu-active if it is intelligently inhibited from being 
incorporated to solutions for some reason like creating a solution that is previously 
visited. Such an attribute is said to be on the tabu-list, which is a main feature of adaptive 
memory structures in tabu search (Fanni and others, 1999:21-32). 

The most common type of short-term memory structure in TS is recency-based 
memory, where the memory stores recently changed solution attributes. Here, attributes 
occurring in solutions recently visited are given a tabu-active status. Solutions, which 
consist of, or have any of these attributes, are put into the tabu-list. Eliminating these 
solutions from the current neighborhood prevents revisiting. 

Long-term memory is generally frequency-based. Here, an array type of memory 
structure, transition (i), represents an n-dimensional counter, which keeps track of the 
number of times an attribute (i) is selected to be part of a solution where n is the number 
of attributes. This gives an opportunity to explore new regions of the solution space 
according to the frequency with which attributes appear in solutions (Glover and Laguna, 
1998:93-122). 
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Attributes, once labeled tabu-active, remain on the tabu-list for a certain number 
of moves, which is their tabu-tenure. Tabu tenure can also vary for different types or 
combinations of attributes, which makes the search dynamic and robust. Empirical 
studies, which are the major information source for TS because of its problem dependent 
feature, show that effective tabu-tenures also depend on the size of the problem instance. 
Here are some guidelines to avoid non-effective tenures: Small tenures can be recognized 
by periodically repeated objective function values. Large tenures miss good solutions. 

In TS terminology, to focus in the neighborhood of good solutions is called 
intensification where the neighborhood is generated by the intersection of good attributes. 
Generally, effective small tenures allow intensifying on solutions close to a local 
optimum while large tenures can catapult the search to some other region of the solution 
space when it is stuck in the neighborhood of a local optimum. To make use of this 
ability is known as diversification (Lokketangen and Glover, 1998:624-658). By 
preventing some moves from being selected for a time period, new regions of the solution 
space are made available for exploration. Diversification can also be obtained by 
restarting or modifying the choice rules of moves. Even simple tabu search methods, 
which most probably have only tabu-tenures as a diversification method, are reported to 
find good solutions. On the other hand, a TS principle called proximate optimality 
principle (pop) says that good solutions can be found next to good solutions. Thus, a 
good TS method is the one that effectively combines intensification and diversification. 

To repeatedly diversify from the feasible region of the solution space to the 
infeasible region and vice versa in successive moves introduces the idea of strategic 
oscillation. Here, making changes in evaluation of the solutions to allow infeasible moves 
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brings changes in neighborhood. This drives the search to cross boundaries between 
feasibility and infeasibility by a selected depth. Controlled strategic oscillation may drive 
the search into regions of the search space not otherwise reachable. 

Within a TS procedure, there are times one wants to override the tabu-active 
status of an attribute. Aspiration criterions facilitate this. The most probable reason to 
override tabu-active status is that a move involving a tabu-active attribute yields a new 
best solution. So, in this case, the improved-best criterion is used to remove the tabu 
classification applied to that attribute. Another example is aspiration-by-default criterion. 
Here, if all available moves are on the tabu-list, a move having fewest tabu-active 
attributes is selected. 

Other dimensions of TS memory structures are quality and influence (Glover and 
Laguna, 1998:125-148). Quality refers to the ability to differentiate between the amounts 
of improvement in solutions. TS uses adaptive memory to keep track of attributes that are 
persistent in many moves {principle of persistent attractiveness ) or common to good 
solutions to produce new solutions of high quality. Candidate lists are used to store these 
good solutions in memory. Quality is also obtained by inducing actions that lead to good 
solutions and penalizing actions that lead to poor solutions (by making use of weights and 
penalty functions). On the other hand, influence measures the degree of change induced 
in a solution. It takes the impact of the choices on the quality as well as structure into 
account that are made during the search. An influential move generally maximizes the 
changes from the current to the next solution. 
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There are a variety of ways to terminate a tabu search. A general termination 
method for TS is to stop if no improvement is detected after a certain number of 
iterations. Other approaches include total number of iterations or total restarts initiated. 

Exploratory Modeling 

“Exploratory modeling is a research methodology that uses computational 
experiments to analyze complex and uncertain systems” (Bankes, 1993:435-449). It can 
be viewed as a sampling over a very large or infinite collection of models that are 
plausible given a priori knowledge. 

Exploratory modeling is an alternative to the traditional approach where all 
known information is consolidated into a single model, which is used to make 
predictions. Taking the amount of uncertainty involved with the real-life problems into 
account, this consolidative model is nothing but a best-estimate model. Consequently, it 
is instable, i.e., small input changes cause big changes in some of the outputs in 
nonmonotonic ways, which brings up questions about the correctness of the model 
(Brooks, Bennett, and Bankes, 1999:67-80). 

Unfortunately, it is too hard to come up with a small valid model with 
uncertainties using the consolidative approach because the uncertainties make validation 
very difficult. One may want to build a very detailed model to reduce the amount of 
uncertainty, but a complex model is not preferred if there are simpler models capable of 
making sufficiently accurate predictions. 

In these cases, a collection of models might be necessary. A simple consolidative 
model can represent the collection of models only if uncertainties are small, since the 
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differences among the properties of the plausible models will be similarly small (Bankes, 
1993:441). However, for other complex systems involving uncertainty to a great extent, 
a large sample from the collection must be examined. This examination, called 
exploratory modeling involves running models drawn from the collection. The selection 
of which model to run depends on the question to be answered. 

A sampling strategy may involve using human judgment to prioritize the 
investigation of the uncertainties. So, the result of an exploratory analysis will be an 
image of the collection that improves gradually as more experiments are run. Finally, the 
analysis provides the most useful results possible based on what is known about the 
problem (Bankes and Gillogly, 1994:353-360). 

In making policy decisions about complex and uncertain problems, exploratory 
modeling can provide new knowledge even where validated models cannot be 
constructed. This depends on the fact that partial information can inform policy even 
when prediction and optimization are not possible. 

An application of exploratory analysis is the weapon mix problem (Brooks, 
Bennett, and Bankes, 1999:67-80). Henning’s paper in the field of aerospace (Henning, 
1999:36-39) and Droogers and Bouma’s paper on soil management (Droogers and 
Bouma, 1997:1704-1711) are other examples of the use of exploratory analysis. 

In general, exploratory analysis allows us to capture the interactions among or 
within any aspects of analysis, including scenario conditions, decisions, or measures of 
effectiveness. Furthermore, exploratory analysis looks from the outside in, generating a 
broad range of outcomes to choose, while a typical sensitivity analysis looks from the 
inside out, from a single-point solution outward around it (Brooks, Bennett, and Bankes, 
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1999:67-80). However, exploratory analysis requires a large amount of computational 
resources. In addition, it does not scale efficiently to multi-scenario situations. 

Other Techniques 

Another technique to approach the multi-scenario optimization problem is robust 
optimization. Robust optimization models search for recommendations that are relatively 
immune to uncertainty in the problem parameters (Bai and others, 1997:895-907). In their 
research, Bai et al ., utilize scenarios to represent uncertainty. These scenarios correspond 
to possible sets of outcomes giving different objective function values. Then, they 
combine these values using weights to create the objective function for the robust 
optimization problem. 

Scenario analysis, a recent technique for modeling and optimizing in the presence 
of uncertainty, employs linear programming to model multi-level nonlinear stochastic 
events (Gafner, 1997). Laferriere conducted scenario analysis to optimize the mixes of 
combat systems under uncertainty (Laferriere, 1997). On the other hand, Doran and Brita 
used scenario analysis to evaluate strategies available to assist the general practitioner in 
reducing smoking behavior among his patients. They claim that scenario analysis 
provides a convenient means to simulate the effects of implementing a range of 
opportunities in isolation or in combination. They further report: ”By defining ‘scenario’ 
in which a number of variables are varied simultaneously, the methodology allows the 
exploration of complex processes on various assumptions” (Doran and Brita, 1998:1013). 

Another technique is stochastic modeling, which is mathematical modeling under 
uncertainty. Any mathematical model where one or more of the coefficients are not fully 
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known at the time of decision making, is a stochastic model. Newton and Guttorp 
presented a statistical analysis providing a stochastic model of early hematopoiesis, 
production of blood cells (Newton and Guttorp, 1995:1146). 

Hill and McIntyre claim that these optimization methods appear limited to 
multiple realizations of a single planning scenario; however they see them as powerful 
techniques for examining uncertainty within a planning scenario. They define a robust 
force structure as that force structure “providing the best overall outcome as evaluated 
with respect to some set of scenarios each of which has an associated likelihood of 
occurrence (Hill and McIntyre, 2000:28). 

Extending these works to come up with a search method for robust multi-scenario 
space as opposed to working in particular scenario projections of exploratory analysis, we 
utilize a ‘parts make the whole’ view in that subspaces generate the multi-scenario space 
according to the likelihood of occurrence. Then, we use scatter search to find a good air 
force structure in the composite multi-scenario space. 

CFAM 


The Combat Forces Assessment Model (CFAM) currently used by the United 
States Air Force, examines force structuring options for a given scenario. This model 
expands capabilities available in its predecessors Heavy Attack, Theater Attack Model 
(TAM), and Mixmaster. 

Heavy Attack assigned values to each target and optimized the total target value 
destroyed (TVD), which was nonlinear. It did not contain budget constraints and did not 
model aircraft attrition. Heavy Attack was the most aggregated of the three models 
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mentioned above, allocating sorties to targets without directly modeling weapons. On the 
other hand, Heavy Attack determined the best weapon and computed effectiveness for a 
sortie. 

TAM was a highly detailed model offering multiple linear objective functions and 
budget and attrition constraints. It optimized globally across time, which made the 
runtime very large. Unlike Heavy Attack, TAM assumed perfect weather knowledge. 

The model objectives required the user to specify goal achievement at a particular time. 

Mixmaster was a hybrid model. The model used a linear program with TVD 
objective function and a greedy sortie allocation scheme. 

Generally, conventional munitions optimization models try to determine the effect 
of having or not having a particular weapon in the inventory ( tradeoffs ), the best way to 
allocate munitions and aircraft to targets, given a fixed inventory and scenario 
( allocation ), and type of weapons needed to meet goals for a given scenario 
(irequirements ) (Yost, 1996:53-71). The models with TVD objective functions need the 
user to specify the desired campaign to be conducted. This means providing the kill 
sequence for the targets while minimizing attrition and resource expenditure. This means 
the view of the user is entered as input to the model. 

To overcome these problems, CFAM provides a choice among objectives. The 
three objectives common to all the models above are maximizing total target value 
destroyed (TVD), minimizing aircraft attrition, and minimizing cost of buying new 
aircraft and weapons. CFAM additionally provides two more objectives: (1) minimizing 
the sum of the penalties associated with not achieving the goals, and (2) minimizing the 
time required to accomplish the phases. The first provides feasibility to the model if the 
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time objectives are not met while the second allows a user definition of overlapping 
between phases. In other words, a phase can start before all the goals in the previous 
phase are met. As stated above, CFAM answers questions about how to employ air assets 
such as the aircraft sorties and equipment needed to destroy specified targets, the time 
needed to reach campaign goals, and aircraft attrition. 
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III. Comparison of GA to Scatter Search 


Introduction 

In this chapter, the genetic algorithm meta-heuristic approach of Bennett is 
compared to our scatter search meta-heuristic. The OptQuest package is used to facilitate 
our approach. To make a comparison, we conducted a 249-run test experiment using 
OptQuest. Our test results provided the basis for continuing the use of OptQuest. The 
following GA discussion and GA results are from Bennett’s study and are reproduced 
here for completeness. See Bennett’s thesis (Bennett, 2000:41-47) for details. 

Genetic Algorithm Meta-Heuristic 

The Experiment 

Each candidate force structure involved five different types of aircraft. The 
maximum number of any strike aircraft included in a candidate force structure was 
limited to 63. The GA employed was the GENEtic Search Implementation System 
(GENESIS Version 5.0) written by John Grefenstette (Grefenstette, 1990:3). Using a 
randomly generated initial population, CFAM was used to evaluate each candidate force 
structure. 

GA Performance Analysis 

Bennett evaluated 942 force structures in three 12-generation GENESIS 
experiments. His 15 best force structures found are shown in Table 1. Note, the GA 
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heavily favored Aircraft 4 and Aircraft 5 while barely utilizing Aircraft 3. A practical 
force structure may require a more balanced force. 


Table 1. Top 15 Force Structures Found By Bennett 


Rank 

Trial 

Number of Aircraft by Type 

Acftl Acft 2 Acft 3 Acft 4 Acft5 

Scenariol 

Fitness Values 
Scenario2 Scenario3 

Total 

1 

181 

15 

4 

0 

56 
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2 

869 

18 

2 

5 

42 

63 

3610.50 

3565.00 

3577.00 

3584.17 

3 

258 

9 

18 

1 

55 

47 

3589.50 

3559.00 

3575.00 

3574.50 

4 

250 

24 

4 

0 

55 

47 

3593.00 

3558.00 

3571.00 

3574.00 

5 

214 

9 

15 

1 

58 

47 

3591.50 

3561.00 

3565.50 

3572.67 

6 

855 

16 

7 

5 

42 

61 

3557.50 

3517.50 

3534.50 

3536.50 

7 

241 

16 

4 

0 

56 

55 

3560.00 

3516.00 

3530.50 

3535.50 

8 

288 

21 

4 

1 

56 

49 

3549.00 

3510.00 

3524.00 

3527.67 

9 

896 

16 

15 

4 

36 

60 

3543.00 

3508.50 

3526.00 

3525.83 

10 

230 

24 

4 

1 

55 

47 

3542.50 

3508.00 

3526.00 

3525.50 

11 

232 

9 

19 

1 

55 

47 

3542.50 

3508.50 

3524.50 

3525.17 

12 

884 

16 

7 

4 

45 

60 

3509.00 

3466.50 

3520.00 

3498.50 

13 

863 

16 

7 

4 

45 

61 

3461.00 

3417.00 

3471.00 

3449.67 

14 

900 

16 

7 

5 

44 

61 

3459.50 

3417.00 

3470.50 

3449.00 

15 

872 

31 

2 

5 

44 

51 

3431.00 

3402.00 

3459.50 

3430.83 


The GA did not find very balanced solutions despite the apparently 
complementary results across each solution. This is likely a result of poor solution space 
coverage in the initial population, along with a low mutation rate and a small number of 
evaluations. Figure 2 plots the initial solutions. Careful observation shows that this 
population is not particularly diverse. 

Three typical GA measures of performance are on-line, off-line, and best so far. 
The data are provided in Table 2 and plotted in Figure 3 for Bennett’s GA Experiment 1. 
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GA Initial Population 



Figure 2. GA Initial Population Coverage of Solution Space 

On-line performance is the average of all evaluations. Off-line performance is the 


average of the current best solutions. The best so far is the overall best solution found. 
We used the measure best so far to compare the two heuristics. 























































All three of Bennett’s experiments gave similar results. Performance figures 
showed the best-so-far curve leveling off while on-line performance curves were still 
increasing. It is our belief that the GA got “stuck” in a local optimum region and could 


not escape. 


Table 2. GA Performance Data for Bennett’s Experiment 1 


Experiment 1 


mgjgi 



imssm 




Average 

0 

Kl!H 

0 



MBBIM 

msatm 

u 

■BBM1 

2 

76 

0 

0 

0.611 

-840 

3278 

3278 

-106 

3 

103 

0 

0 

0.639 

-541 

3278 

3278 

323 

4 

131 

0 

0 

0.64 

-468 

3295 

3358 

-53 

5 

155 

0 

i 

0.671 

-225 

3307 

3371 

852 

6 

182 

1 

i 

0.694 

-9 

3319 

3681 

1229 

8 

227 

2 

2 

0.716 

187 

3391 

3681 

1481 

9 

249 

2 

2 

0.721 

340 

3417 

3681 

1966 

10 

270 

2 

3 

0.718 

469 

3437 

3681 

2047 

12 

315 

3 

4 

0.746 

689 

3472 

3681 

2136 


Scatter Search Meta-Heuristic 

Solution Structure 

Similarly, we also used a candidate force structure consisting of five different 
types of aircraft for ease of comparison to the GA meta-heuristic. For comparison with 
the GA approach, the maximum number of any strike aircraft included in a force 
structure aircraft type is again limited to 63. A sample solution to the problem would be 
represented as follows: 

V,= (2 0 16 8 10). 

Ai A2 A3 A4 Aj 
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Vj is a force structure consisting of Aj = two type-one aircraft, A 2 = zero type-two aircraft, 
,4^ = 16 type-three aircraft, A 4 = eight type-four aircraft, and As = ten type-five aircraft. 

Evaluation 

The particulars of the Scatter Search are embedded in the algorithm, OptQuest CL 
prepared by Laguna (OptQuest Callable Library for C Applications, Optimization 
Technologies, Inc). Similar to GENESIS, this algorithm allows a user to build an 
application to search for the optimal solution to his problem using the “black-box” 
approach to evaluating the objective function. The application uses the objective function 
evaluator, CFAM, to measure the quality of the candidate solutions generated by the 
OptQuest CL routines. 

An important feature of OptQuest CL is that it uses the variable ranges and the 
constraints to determine the feasibility of the solution before the solution is sent to the 
objective function evaluator, CFAM. The OCLSetBoundFreq function sets the frequency 
parameter for the boundary search strategy. The boundary search strategy is a mechanism 
to direct the search towards the feasibility boundary defined by the bounds on the 
decision variables. The default value is 0.5, which means that the strategy to drive the 
values of the decision variables towards the boundary is used 50% of the time when new 
solutions are generated. If good solutions are likely to fall at or near the feasibility 
boundary, it is recommended to increase the value of this parameter. Conversely, if good 
solutions are suspected to lie near the middle of the variable range (when no constraints 
are defined), it is recommended to decrease the default value for the boundary strategy. 
After running 400 experiments shown in Appendix 2, we decided to use 0.3 as the 
boundary strategy, which gave the highest objective function value and a good average. 
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Sactter Search Performance Analysis 


A total of 249 force structures were evaluated. Using the same performance 
measures from Genetic Algorithms and counting 20-run blocks as generations, we found 
very good results, which are more than two times better than the GA results. This is a 
probable consequence of OptQuest’s ability to intelligently sample the solution space 
during initialization. Figure 4 can be compared to Figure 2. The OptQuest initial 
solutions seem to offer greater diversity than the GA initial solutions. While the GA 
generates random initial candidate structures, OptQuest employs a systematically defined 
initial population. 

Scatter Search Initial Population 



Figure 4. Scatter Search Initial Population Coverage of Solution Space 


Unlike the GA, Scatter Search favors Aircraft 4 and Aircraft 5 only slightly. The 
best force structure, trial 22, which consists mainly of the first 3 types of aircraft, 
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supports this idea, as seen in Table 3. More balanced structures, such as trials 239 and 
149, also have high objective function values. Furthermore, Scatter Search utilizes 
Aircraft 3 better, as opposed to GA’s poor utilization. 


Table 3. Top 15 Force Structures Found By OptQuest 


Rank 

Trial 

Aircraft 
A1 A2 

Number 

A3 

by 

A4 

Type 

A5 

Fitness Values 

Scenariol Scenario2 Scenario3 

Average 

Fitness 

1 

22 

11 

10 

13 

1 

0 

8253.5 

8323 

8364.5 

8313.667 

2 

14 

2 

4 

6 

27 

1 

8044.5 

8034 

8087.5 

8055.333 

3 

196 

3 

6 

5 

26 

9 

7589.5 

7564.5 

7626 

7593.333 

4 

53 

0 

0 

0 

9 

55 

6868.5 

6837 

6890 

6865.167 

5 

237 

0 

0 

0 

2 

4 

9720.5 

-300.5 

9752.5 

6390.833 

6 

194 

1 

1 

2 

9 

0 

-702.5 

9322 

9343.5 

5987.667 

7 

140 

4 

31 

3 

17 

28 

5934.5 

5918 

5964 

5938.833 

8 

248 

0 

7 

1 

5 

3 

-780.5 

9266.5 

9321.5 

5935.833 

9 

43 

5 

6 

1 

27 

47 

5779.5 

5733.5 

5790.5 

5767.833 

10 

10 

0 

0 

0 

7 

12 

9045 

-976.5 

9078 

5715.5 

11 

239 

1 

4 

3 

5 

7 

-1008 

8989 

9010.5 

5663.833 

12 

149 

17 

24 

18 

17 

17 

5423 

5371 

5460.5 

5418.167 

13 

200 

1 

3 

4 

19 

4 

-1596 

8426 

8442.5 

5090.833 

14 

11 

3 

0 

1 

32 

63 

5078 

5034.5 

5057 

5056.5 

15 

245 

7 

1 

4 

6 

18 

-1996 

8244.5 

8298.5 

4849 


Best-so-far and on-line curves are not generally available via OptQuest but were 
derived from the data for comparison purposes. 

Table 4 and Figure 5 shows the best-so-far curve leveling off. However, the on¬ 
line performance curve is still increasing, as it was in GA approach. This warrants 
further investigation. Many more runs are required before coming to any firm 
conclusions. 
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Figure 5. Scatter Search Performance for Test Experiment 


Results 


Both the GA and OptQuest employ a population-based heuristic available in 
software amenable to our multi-scenario optimization problem. Because our ‘black-box’ 
function evaluator uses CFAM, each evaluation is computationally expensive. Because of 
this computational expense, we want our optimizing meta-heuristic to make use of the 
functional evaluations. 

The test results favor OptQuest over the GA. The OptQuest results show higher 
total fitness values and greater diversity among the solution values. Further, these early 
OptQuest solutions appear more balanced than the early GA solutions with respect to 
usage of each aircraft type. 

The test results also point out a potential problem with the total fitness function. 
While the fitness values for the GA solutions do not seem to vary much by scenario, this 
is not the case for OptQuest solutions. Clearly, a truly robust solution must be adequate in 
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every scenario in addition to best overall. Further, the fitness function appears to contain 
some bias favoring excludingly small force structures. Clearly, reality necessitates larger 
force structures than trial 237 in Table 3 might suggest. 


The next chapter formally presents the scatter search methodology. 


Table 4. Scatter Search Performance Data 




Fitness Values 

Generation 

Trials 

Online 

Offline 

Best 

Average 

1 

21 

-2993.81 

8055.333 

8055.333 

-2447.9 

2 

42 

-2143.31 

8184.5 

8313.667 

-1153.02 

3 

63 

-1718.64 

8227.556 

8313.667 

-1101.29 

4 

84 

-1841.26 

8249.083 

8313.667 

-2385.57 

5 

105 

-1786.97 

8262 

8313.667 

-1655.94 

6 

126 

-1910.79 

8270.611 

8313.667 

-2845.02 

7 

147 

-1887.14 

8276.762 

8313.667 

-1292.4 

8 

168 

-1865.86 

8281.375 

8313.667 

-1618.65 

9 

189 

-1733.33 

8284.963 

8313.667 

-643.595 

10 

210 

-1558.45 

8287.833 

8313.667 

29.56349 

11 

230 

-1438.93 

8290.182 

8313.667 

-406.968 

12 

250 

-1336.6 

8292.139 

8313.667 

-828.5 
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IV. Methodology and Sensitivity Analysis 


Introduction 

In this chapter, we focus first on the methodology of employing an advanced meta¬ 
heuristic search procedure, i.e. Scatter Search, to guide the use of the existing Air Force 
Studies and Analyses Agency’s (AFSAA) combat model CFAM to evaluate the 
effectiveness of candidate force structures across the multi-scenario space and terminating 
with robust solutions of AEF strike force structures. We then focus on analyzing the results 
of our experiment. 

Methodology 

Our approach is based on Hill and McIntyre’s methodology for robust, multi¬ 
scenario optimization (Hill and McIntyre, 2000:27). Defining robust force structure as a 
“force structure providing the best overall outcome as evaluated with respect to some set of 
scenarios each of which has an associated likelihood of occurrence”, they suggest using a 
controller interface to perform necessary interactions between the meta-heuristic and the 
combat model. The controller provides the model (CFAM) with candidate solutions 
generated by the meta-heuristic. CFAM evaluates these solutions. The controller takes the 
evaluations, or measures of effectiveness (MOE), combines different scenario MOEs for a 
specific force structure into a single measure of merit (MOM), i.e. a weighted objective 
function value, and provides them to the meta-heuristic to continue the search of 
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unexplored solution space regions. The cycle stops after a specified number of iterations or 
when convergence to an optimum is detected. 



Figure 6. Robust Multi-Scenario Optimization Methodology 


The methodology specifics applied to our problem are as follows (Figure 6): The 
controller interface is a combination Visual Basic/DOS routine playing the role of a data 
manager. The employed heuristic method is a Scatter Search Algorithm, which guides the 
search examine different force structures. CFAM is our combat model used to evaluate the 
force structures suggested by the Scatter Search. 

The controller serves four functions. First, the controller routine makes necessary 
changes in the CFAM input files to accommodate the force structure provided by the 
Scatter Search. Second, it runs CFAM for the input force structure. Then it combines 
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CFAM results into the weighted objective function value. Finally, it triggers the Scatter 
Search again with this value. 

Scatter Search Meta-Heuristic 

As discussed in the previous chapter, due to the computational expense of the 
objective function evaluations, we preferred a search method that determined the feasibility 
of the solutions before they are sent to the evaluator, CFAM. In addition, our test results 
showed that Scatter Search gave better and more balanced force structures. 

Scatter Search generates and places the candidate solutions into a text file. The file 
is read by the controller interface which makes the necessary changes in the CFAM input 
files, runs CFAM for each scenario, combines these outputs (each scenario) into a single 
objective function value and passes this value to OptQuest via a test file. 

Combat Forces Assessment Model (CFAM) 

CFAM is currently used by United States Air Force for evaluating force structuring 
options for a given scenario. CFAM is a large-scale linear program (LP) used as an 
analytical tool for determining the impacts of budget, attrition, force structure, targeting 
decisions, and munitions inventories on war fighting capabilities (AFSAA, 1997:3). 

CFAM produces its outputs, whenever the campaign to meet a scenario objective 
completes or seven time periods have elapsed. Five of the standard CFAM outputs are 
used to produce Bennett’s MOE to evaluate the force structures (Bennett, 2000:39). The 
MOE is a weighted objective function, which we used for ease of comparison. The weights 
assigned to each output are based on opinion. The five CFAM outputs used are: final phase 
of campaign encountered, remaining targets in the current campaign phase, campaign 
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completion, the time period when the scenario was complete, and total number of aircraft 
lost. These combine to produce an objective value for each scenario. Similar weights are 
used to combine the objective function values from each scenario to construct Bennett’s 
multi-scenario objective function designed to find robust force structures. The campaign 
phases are: 1) SEAD (suppression of enemy air defense), 2) HALT (halting enemy 
advancement), 3) ATTRIT (attrition of enemy forces), and 4) the counter offensive. 

CFAM comes in two variants. Quick Strike allows the user to optimize within 
distinct time periods of an air campaign, while Time Strike optimizes across an entire 
campaign (AFSAA, 1997:4). We used Quick Strike to find the parameters needed to 
calculate the multi-scenario objective function value of a candidate solution. 

The methodology is applied against Bennett’s three notional scenarios derived from 
the unclassified CFAM training database. The weapons inventory remained constant in the 
7-day-long scenarios, which differed slightly in favoring ground targets to air assets. Our 
focus was on the allocation of five types of aircraft, again derived from Bennett’s database 
by enhancing the distinctions in surge rate, minimum altitude limit, and minimum 
estimated kills per sortie. The parameters of baseline aircraft are shown in Table 5. 
Bennett’s CFAM setup is given in Appendix 8. For more detailed information see Bennett 
( 2000 ). 

The Controller Interface 

All the interactions between CFAM and OptQuest are done through a batch file 
called master, hat (Appendix 7). This file, written originally by Bennett, is used here for the 
comparison of two heuristic search methods, Genetic Algorithms and Scatter Search. The 
file is called from the user-written evaluation method of OptQuest (Appendix 1), where it 
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receives the candidate force structure and returns the evaluation after executing CFAM 
once for each scenario and combining the results into the single weighted objective 
function value. 


Table 5. Bennett’s Candidate Aircraft Types and Characteristics 


ID 

Description 

Sortie 

Surge 

Rate 

Min 

Alt (ft) 

Min EKS 

1 

Aircraft 1 

2 

9999 

0.005 

2 

Aircraft 2 

2 

9999 

0.005 

3 

Aircraft 3 

2 

9999 

0.005 

4 

Aircraft 4 

2 

9999 

0.005 

5 

Aircraft 5 

2 

9999 

0.005 

6 

Aircraft 6 

2 

9999 

0.005 

7 

Aircraft 7 

2 

9999 

0.005 

WfmTW: 

WEEK.- 

Piiiss 

1000 

0.02 

|F 

Aircraft 9 

iisiiii 

600 

ifebSil 

10 



f 600 

HSGKOT^! 

11 

Aircraft 11 

2 

9999 

0.005 

12 

Aircraft 12 

2 

9999 

0.005 

13 

Aircraft 13 

2 

9999 

0.005 

14 

Aircraft 14 

2 

9999 

0.005 

15 

Aircraft 15 

2 

9999 

0.005 

16 

Aircraft 16 

2 

9999 

0.005 

17 

Aircraft 17 

2 

9999 

0.005 

r.%i8 f 

Aircraft 18 


9999 

0.005 

19 

Aircraft 19 

siii 

iti6ooi#i 

te).0lSl; 


Within the body of the controller interface, user defined weights and CFAM output 
parameters enable us to calculate a value for our scenario objective function, which is the 
same as Bennett’s (Bennett, 2000:39): 

O j = - W]D 0 + w 2 j D , + w 3 Z) 2 - w 4 j D 3 - w 5 D 4 - w 6 Atotal (l) 


where 


w 1; through w 6j are parameter weights defined by the decision maker 
Do = time period when campaign ended (ranging from 1 to 7), 
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Di ^campaign completed (0 for no, 1 for yes), 

D 2 = current phase when campaign ended (ranging from 1 to 4), 

D 3 = total aircraft lost, 

D 4 = targets remaining for current phase, 

Atotal = the total number of aircraft in the force structure. 

Bennett used equal parameter weights for each scenario and put importance on completing 
the campaign, as well as any campaign phase. He penalized big force structures, long 
campaign completion times, large amounts of aircraft losses and large numbers of 
remaining targets. We discuss some of the drawbacks of this set up in the sensitivity 
analysis section. Bennett further calculated the multi-scenario objective function value for 
force structure y, as: 

i 

v; = 2 >, (2) 

/=1 

where i is the scenario number, y is the number of the candidate force structure, and P, is 
the probability of scenario i occurring. The parameter weights and probability of 
occurrence defined by Bennett and used here are: 

w h =50, w 2 . =10000, w 3 . =125,w 4j =50,w 5j =50, w 6 . = 50, and = 1/3 for i = 1, 2, 3 
(Bennett, 2000:40). 

Sensitivity Analysis 

The experiments were run on a notebook computer with 256 megabytes of random 
access memory running under Windows 98 with a 366 MHz Pentium II processor. The 
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notional scenarios were constructed using CFAM version 2.5 and evaluated using GAMS 
2.5 linked with the CPLEX Linear Optimizer version 6.5.2. The Scatter Search guiding the 
search procedure was OptQuest version March 3, 2000, compiled using Microsoft Visual C 
6 . 0 . 

A total of 689 force structures were evaluated in a single OptQuest experiment. The 
data obtained from these 689 solutions can be found in Appendix 4. For ease of comparison 
to Bennett, the best 15 force structures found are extracted from this list and shown in 
Table 6. Unlike the GA results, structures found by OptQuest seem to cover a larger 
portion of the solution space. We conjecture that the GA with its similar structures was 
stuck in a local optimum due to an ineffective initial population. OptQuest, however, 
screening out the solutions that are likely to be poor without even sending to the CFAM, 
does eliminate the problems facing GA concerning solution space coverage. This idea is 
supported by the great diversity in the solutions table, compared to GA’s best solutions 
table in the previous chapter. 
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Table 6. Best 15 Solutions Found by 689-run Experiment 


Rank 

Trials 

Aircraft 

A1 A2 

Number 

A3 

by 

A4 

Type 

A5 

Fitness Values 

Scenariol Scenario2 Scenario3 

Total 

Fitness 

1 

226 

10 

0 

0 

5 

0 

9333.5 

9321.5 

9371.5 

9342.167 

2 

462 

7 

8 

7 

8 

0 

8576.5 

8530 

8576.5 

8561 

3 

447 

10 

1 

2 

8 

15 

8270.5 

8231 

8284.5 

8262 

4 

249 

7 

10 

10 

10 

2 

8100 

8070 

8112.5 

8094.167 

5 

14 

2 

4 

6 

27 

1 

8044.5 

8034 

8087.5 

8055.333 

6 

651 

10 

16 

16 

12 

3 

7211 

7163.5 

7255 

7209.833 

7 

679 

0 

0 

52 

5 

1 

7145.5 

7120.5 

7181.5 

7149.167 

8 

680 

0 

0 

63 

0 

0 

6933.5 

6917.5 

6968 

6939.667 

9 

685 

4 

5 

51 

3 

0 

6911 

6868 

6954 

6911 

10 

53 

0 

0 

0 

9 

55 

6890.5 

6842 

6895 

6875.833 

11 

54 

26 

4 

11 

9 

15 

6796 

6781.5 

6834 

6803.833 

12 

327 

3 

1 

1 

0 

0 

-339 

9718.5 

9739 

6372.833 

13 

406 

4 

0 

2 

0 

0 

-369 

9670 

9691 

6330.667 

14 

372 

3 

1 

4 

0 

0 

-485 

9567.5 

9587.5 

6223.333 

15 

373 

2 

8 

24 

26 

17 

6219.5 

6184.5 

6223.5 

6209.167 


Bennett’s concern about lowest level of organic maintenance support for aircraft, 
i.e. squadron level, also arises in our situation. Taking into account that a fighter squadron 
usually consists of 18 or 24 aircraft, employing a portion of a squadron clearly is not the 
right decision because of the rising cost. To avoid this problem, more realistic force 
structures can be selected from the best solutions table. This flexibility in meeting different 
needs is provided by the robustness of our approach. 

On the other hand, another concern not applicable to GA results has emerged, as 
test results also pointed out earlier. The objective function tolerates unrealistically small 
structures, as seen in trials 327,406 and 372, having at most just 8 aircraft in the force 
structure. The penalty applied to parameter Atotal for finding smaller force structures 
should be revised. In addition, the function should also avoid assigning high values to force 
structures, which have obtained a low value in one of the scenario evaluations. The above 
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trials, 327, 372 and 406, also provide examples for this case. The decision maker clearly 
would not employ an expeditionary force that meets the objectives of two scenarios, but 
completely fails in one of the probable crisis situations it would face. 

Another indication of the test results was that Scatter Search did not favor any one 
of the 5 types of the aircraft, while Bennett’s GA heavily favored Aircraft 4 and Aircraft 5. 
It also used a balanced selection method of the aircraft. His GA barely utilized Aircraft 3. 
For multi-scenario optimization problems, the superiority of Scatter Search to GA is 
verified by the 689-run experiment. 

To find robust force structures to our multi-scenario optimization problem, we used 
an approach similar to Bai, et al (Bai and others, 1997:898). By introducing weights and 
combining them, we generated a multi-scenario objective function value across the range of 
uncertainty represented by the scenarios. 

Exploratory analysis approach is also applicable to our problem. To relieve the 
concern about employing a portion of a squadron, we have the option to select the more 
suitable force structure among the best solutions found in order to meet our needs. 

To meet the definition of robustness introduced by Ignizio, we need to find force 
structures whose objective function values are relatively immune to the slight changes in 
the original structure (Ignizio, 1999:6). In order to do this, we performed a regression 
analysis using JMP IN Statistical Discovery Software Versions 3.0 and 4.0. The details of 
the analysis are given in Appendix 5. We fitted a regression function to our experiment 
data and searched the solution space for robust regions with the guidance of this function. 

To perform the search, we examined the neighborhood of the stationary points of the fitted 
function. However, the data was so spread out that fitting a regression function was 
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smoothing all the variability in the data and specifying the local optima as outliers. The 
approach was likely to bring robustness but its necessitation of sacrificing optimality made 
us look for alternative ways to find an answer to the robust multi-scenario optimization 
problem. We also regressed over only the local optima, and were able to find a stationary 
point for Aircraft 3, but the function was not robust in the neighborhood. 

An alternative approach was looking at the plot of the experiment results in the 
solution space through a hyperplane defined by any two of the aircraft. To obtain a clear 
picture of the hyperplane, we averaged the data points over the number of aircraft (Figure 
7). These figures show that it is still hard to find a relatively smooth region providing 
robust solutions because of the rugged nature of the solution space. 

For further examination, we used the average objective function value by type of 
aircraft table (Appendix 6), which provided the basis for Figure 7. The relevant portion of 


Multi-Scenario 
Objective Function 
Value 



Figure 7. Average Objective Value of Aircraft 1 vs. Aircraft 2 
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this table is given in Table 7. The results supported our regression analysis: The robust air 
expeditionary force should be constructed in the neighborhood of 7 Aircraft Type 4, which 
is the exact same result obtained by the regression approach. The force structure should 
also have 7 Aircraft Type 1, in the neighborhood of the regression result of 5 aircraft. 


Table 7. Relevant Portion of the Objective Value Average Table 


Number of 
Aircraft 

Tyi 

pe of Aircraft 

A1 

A2 

A3 

A4 

A5 

0 

- 32.11742424 

595.6432 

127.8956386 

- 602.596 

212.3323 

1 

- 620.5530303 

805.3023 

974.8809524 

- 2176.33 

531.1296 

2 

- 62.88571429 

1092.8 

-121.4733333 

- 1326.79 

-245.548 

3 

38.72727273 

- 457.8 

174.9416667 

- 374.471 

32.17778 

4 

- 371.6621622 

205.9097 

- 97.58928571 

- 775.897 

735.25 

5 

- 220.2356322 

508.4386 

- 752.6018519 

1073.56 

- 2034.32 

6 

-127.3133333 

- 367.765 

- 130.5238095 

52.4023 

- 1804.62 

7 

191.5944444 

- 1414.72 

619.1875 

557.0625 

-4146 

8 

-273.3733333 

- 156.167 

- 544.7651515 

629.7105 

- 2462.23 

9 

- 549.3431373 

- 192.833 

- 581.8181818 

- 520.005 

- 4707.53 

10 

388.7971014 

365.625 

- 358.8452381 

570.1739 

- 3346.88 

11 

948.7111111 

- 1055.29 

- 528.1666667 

- 1033.01 

- 3167.42 

12 

-2512 

492.8333 

- 1360.833333 

793.75 

- 3171.47 

13 

- 290.2708333 

- 1944.92 

- 132.8333333 

- 1038.55 

- 3821.13 

14 

- 367.6212121 

- 992.667 

134.5606061 

- 1732.01 

- 5578.44 

15 

142.1833333 

- 1941.6 

- 155.5333333 

- 775.083 

4414.556 

16 

- 660.6060606 

591.0714 

- 176.0625 

- 1417.89 

- 2543.08 

17 

- 34.375 

- 1178.35 

- 4926.619048 

- 911.716 

- 255.444 

18 

- 368.3181818 

- 2566.22 

- 2837.916667 

1507.808 

- 1231.88 

19 

- 1475.65 

- 1143.3 

- 2002.333333 

- 2136.88 

- 1681.88 

20 

- 362.1041667 

- 926.375 

- 2286.125 

317.9444 

- 2302.28 

21 

- 521.9761905 

98.84848 

- 1198.515152 

- 1841.1 

- 1789.98 

22 

- 2375.384615 

- 536.806 

- 2586.666667 

- 5.52083 

- 3593.89 

23 

- 3204.083333 

- 1941.14 

- 1707.5 

- 1512.31 

303.7667 

24 

- 1023.583333 

651.1458 

1171.458333 

1237.389 

- 8055.17 

25 

- 2862.766667 

- 2405.61 

- 601.6666667 

1266.028 

- 2434.58 

26 

- 1796.962963 

- 933.796 

- 1600.333333 

- 388.296 

- 2577.2 

27 

- 925.7 

- 2525.25 

- 1116.638889 

38.95833 

- 1213.08 

28 

- 5230.958333 

- 1414.03 

505.1666667 

- 120.5 

- 1670.75 

29 

- 3203.066667 

1895.083 

2321.25 

- 663.7 

- 1824.08 

30 

- 1238.916667 

118.75 

330.25 

- 1857.87 

- 2319.57 

31 

- 342.9166667 

- 374.867 

- 2661.916667 

- 200.667 

- 767.3 

32 

- 1996.722222 

- 2762.42 

- 3557.681818 

- 1366.32 

- 3335.61 
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These results are also verified by the best solutions table having 8 and 9 Aircraft Type 4, in 
4 out of 15 structures; 7 and 10 Aircraft Type 1, in 5 out of 15 structures (Table 6). The 
analysis further points out that we should have around 1 Aircraft Type 2 (0 and 1, in 8 out 
of 15 also in Table 6). 

There is an indication of robustness in regions where Aircraft Type 3 is utilized at 
levels of 0-3 and 28-30 aircraft. However, this area is smoothed in our local optima 
regression, which favored 18 aircraft. Finally, Aircraft Type 5 has relatively stable values 
in the range 0-4 as verified by the best solutions table 11 times. 
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V. Findings and Conclusions 


Findings 

As widely discussed in the previous chapters, the use of Scatter Search meta¬ 
heuristic to guide the quest for optimality in the non-convex solution space of our multi¬ 
scenario optimization problem has performed dramatically better than the use of Genetic 
Algorithms (as seen by the great difference in results obtained). Integrating simulation 
and optimization, and employing a neural accelerator to screen out solutions of probable 
poor quality, OptQuest has proved to be a powerful tool to examine solution spaces of a 
huge and complex nature. 

The robust optimization technique of incorporating uncertainty into a weighted 
objective function and the exploratory analysis approach of providing near optimal 
solutions to the decision maker to trade off in deference to other factors, melded a 
successful research methodology. This methodology also gives the flexibility of 
changing the problem setup to meet different needs and concerns, which can be 
performed in two ways: (1) Changing CFAM inputs, (2) Changing the weights of the 
associated parameters in the scenario objective function. In our study for example, the 
weight for parameter Atotal should be changed to avoid favoring exclusively small air 
expeditionary force structures. CFAM also presents a variety of options. In addition, the 
range of the available number of aircraft can also be changed in the evaluation method. 

We have obtained good results using CFAM, the latest combat model to build 
force structures to respond to a single theater scenario, as the evaluator in our study. 
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However, CFAM also fits in the category of models whose runs are computationally 
expensive. The processing times will become even longer when the methodology is 
applied to real scenarios, which will most probably try to allocate more than five types of 
aircraft. 

Conclusions and Areas for Further Research 

Our analysis showed that the multi-scenario objective function developed by 
Bennett (Bennett, 2000:40) needs to be revised. As mentioned in the previous chapter, the 
current objective function allows exclusively small force structures to find a place among 
the best solutions. Under normal conditions, there will not be a pilot in the world who 
would want to be a part of an air expeditionary force structure consisting of only 5 
aircraft as trial 327 in Table 6 suggests. Besides, the current function assigns a high value 
to the force structures performing well in responding to two of the scenarios but fairly 
bad in responding to the other one. Another drawback of the function is that it gives a 
very rugged solution space full of local optima, leaving very little space for robust 
plateaus if any at all. Clearly, the definition of ‘robust’ is about having values that occupy 
some relatively flat space, or plateau, across changing input parameters. 

Our methodology performed well with respect to notional scenarios. The 
objective is, however, to be applicable to real scenarios. Before attempting to use 
classified data to model realistic scenarios, we suggest some refinements like the ones 
described above and further testing on a lot more capable computers to reduce the 
processing time. To address real life problems, specialized CFAM input files will also be 
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necessary. Another area for further research is weapons allocation in addition to aircraft 
allocation at more complicated levels. 

Future efforts should focus on discovering alternative ways to determine various 
weights of the scenarios. Enough consideration should be given to whether or not there 
are other methods to perform a thorough sensitivity analysis to address this problem. If 
there are, the question about the method of employment should be given an adequate 
answer. 
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Appendix 1: Scatter Search Code with Evaluation Function 


/* File: scattersearch.c 

*/ 

#include "ocl.h" 

#include <stdio.h> 

#include <stdlib.h> 

#include <math.h> 

#include <string.h> 

#include <fcntl.h> 

#include <io.h> 

/* 

CFAM Multi-Scenario Evaluation Function for OptQuest 

Written By: Gokay Bulut 

Language: C/C++ 

Sources used: C/C++ manuals 
OptQuest manual 
Eval.c (Bennett, 2000:51) 

*/ 

#defme NUM_VARIABLES 5 

double evaluate(double *); 

int main(int argc,char *argv[]) 

{ 

double DecVar [NUM_V ARI ABLES+1 ]; 
double ObjVal, Boundary; 
long Example, nsol, status; 
int i, Totallter; 

if (argc != 3) 

{ 

printf("usage: scattersearch <total-iter> <boundary>\n"); 
exit(l); 

} 

Totallter = atoi(argv[l]); 

Boundary = atof(argv[2]); 
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/* Allocating memory */ 

Example = OCLSetup(NUM_VARIABLES,0,0,"MAX", 536831); 
if (Example < 0) { 

printf("OCLSetup error code %d\n". Example); 
exit(l); 

} 

/* Defining variables */ 

for (i = 1; i <= NUM_VARIABLES; i++) { 

status = OCLDefineVar(Example, i, 0, OCLNULL, 63,"DIS", 1); 
if (status < 0) { 

printf("OCLDefineVar error code %d\n", status); 
exit(l); 

} 

} 

/* Set parameters and initialize the population */ 
status = OCLSetSolutions(Example, Totallter); 
if (status < 0) { 

printf("OCLSetSolutions error code %d\n", status); 
exit(l); 

} 

status = OCLSetBoundFreq(Example, Boundary); 
if (status < 0) { 

printf("OCLSetBoundFreq error code %d\n", status); 
exit(l); 

} 

status = OCLInitPop(Example); 
if (status < 0) { 

printf("OCLInitPop error code %d\n", status); 
exit(l); 

} 

/* Generate and evaluate Totallter solutions */ 
for (i = 1; i <= atoi(argv[l]); i++) 

{ 

nsol = OCLGetSolution(Example,DecVar); 
if (nsol < 0) { 

printf("OCLGetSolution error code %d\n", nsol); 
exit(l); 

} 

ObjVal = evaluate(DecVar); 

status = OCLPutSolution(Example,nsol,&ObjVal,(double*)OCLNULL); 
if (status < 0) { 

printf("OCLPutSolution error code %d\n", status); 
exit(l); 
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} 

if (!(i%1000)) { 

status = OCLGetBest(Example,DecVar,&ObjVal); 
if (status < 0) { 

printf("OCLGetBest error code %d\n", status); 
exit(l); 

} 

printf("Best solution after %7d iterations is %9.6f\n",i,ObjVal); 

} 

} 

/* Display the best solution found */ 

status = OCLGetBest(Example,DecVar,&ObjVal); 

if (status < 0) { 

printf("OCLGetBest error code %d\n", status); 
exit(l); 

} 

printf("\n"); 

for(i=l ;i<=NUMVARIABLES;i++) printf("x[%2d] = %9.61f\n",i, DecVar[i]); 

/* Free memory */ 

status = OCLGoodBye(Example); 

if (status < 0) { 

printf("OCLGoodBye error code %d\n", status); 
exit(l); 

} 

return 0; 

} 


/* Evaluation function */ 
double evaluate(double *vect) 


{ 

FILE * Instream, *Outstream, *fopen(); 
float sumf; 
double sum; 


Print candidate force structures suggested 
by OptQuest to the file ‘numac.txt’ 

Outstream = fopen("numacft.txt", "w+"); 

fprintf(Outstream,"%5.0P/o5.0f%5.0f%5.0f%5.0f\n",vect[l],vect[2], 
vect[3], vect[4], vect[5]); 
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fclose(Outstream); 

printf("% 8 .3 f% 8 .3 f% 8 .3 P/ 08 .3 f% 8 .3 fvn" ,vect [ 1 ], vect[2], vect[3], 
vect[4], vect[5]); 


/* 

*/ 


Call controller interface batch file for CFAM evaluations 
system("master"); 


/* Read the multi-scenario objective function value 

from the file ‘evalue.txt’. 


*/ 


Instream = fopen("evalue.txt", "r"); 

fscanf(Instream, "% 10 f", &sumf); 

sum=sumf; 

fclose(Instream); 

return (sum); 
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Appendix 2: CFAM Evaluation Runs for Boundary Strategy Selection 
Experiment 1, with Strategy 0.2 


Aircraft Number by Type Fitness Value Total 


Trials 

A1 

A2 

A3 

A4 

A5 

Scenariol 

Scenario2 

Scenario3 

Fitness 

sorted 

1 

32 

32 

32 

32 

32 

2014 

- 8024.5 

2025 

- 1328.5 

4644.833 

2 

0 

0 

0 

0 

0 

- 267.5 

-277 

-250 

- 264.833 

3766.167 

3 

63 

63 

63 

63 

63 

- 5665.5 

-5679 

-5631 

- 5658.5 

3484.333 

4 

61 

58 

3 

41 

1 

1911.5 

- 8171.5 

- 8150.5 

- 4803.5 

2854 

5 

8 

62 

5 

34 

56 

1703 

-8296 

- 8293.5 

- 4962.17 

2723.333 

6 

8 

62 

5 

34 

56 

1828 

1784.5 

1837 

1816.5 

2546.667 

7 

59 

6 

37 

2 

52 

2235 

-7815 

2240.5 

- 1113.17 

2385.167 

8 

58 

60 

49 

3 

38 

-395 

-10441 

- 344.5 

- 3726.83 

2324.833 

9 

58 

60 

49 

3 

38 

- 382.5 

-10425 

-376 

- 3727.83 

2291.833 

10 

57 

14 

41 

17 

43 

1395 

- 8626.5 

1428 

- 1934.5 

1816.5 

11 

61 

0 

33 

0 

61 

2281 

-7773 

2278.5 

- 1071.17 

1811.167 

12 

36 

63 

63 

63 

4 

-1408 

- 11460.5 

-1401 

- 4756.5 

1767 

13 

50 

9 

42 

18 

34 

2334 

- 7693.5 

2398.5 

-987 

1158.167 

14 

63 

3 

32 

0 

63 

1969 

-8078 

1982.5 

- 1375.5 

1137.833 

15 

11 

16 

60 

63 

0 

2543.5 

-7513 

2592 

- 792.5 

771.3333 

16 

59 

21 

33 

19 

49 

927 

- 9087.5 

-9070 

- 5743.5 

585.5 

17 " 

59 

0 

41 

0 

55 

2286.5 

2266.5 

2322.5 

2291.833 

365.3333 

18 

61 

63 

0 

58 

0 

937 

-9113 

943 

-2411 

214.1667 

19 

44 

5 

34 

8 

52 

2821 

- 7193.5 

2848.5 

-508 

156 

20 

63 

7 

40 

0 

52 

1937 

- 8105.5 

1949.5 

- 1406.33 

- 68.6667 

21 

0 

0 

0 

31 

55 

5734 

-4316 

5737.5 

2385.167 

- 217.833 

22 

50 

14 

35 

3 

52 

2413.5 

-7666 

-7650 

- 4300.83 

- 264.833 

23 

63 

0 

39 

1 

52 

2285.5 

- 7764.5 

2291.5 

- 1062.5 

- 369.167 

24 

23 

40 

30 

33 

32 

2134 

-7915 

2141.5 

- 1213.17 

- 484.667 

25 

23 

40 

30 

33 

32 

2115 

- 7927.5 

2126 

- 1228.83 

-508 

26 

63 

6 

46 

3 

63 

946.5 

- 9069.5 

- 9052.5 

- 5725.17 

- 606.333 

27 

60 

21 

38 

25 

54 

144.5 

134 

189.5 

156 

- 626.833 

28 

58 

0 

36 

0 

50 

2866.5 

2821.5 

2874 

2854 

-710 

29 

42 

25 

26 

13 

53 

2081.5 

-7966 

2094 

- 1263.5 

- 783.5 

30 

25 

43 

16 

23 

55 

1939 

- 8115.5 

1936.5 

- 1413.33 

- 792.5 

31 

7 

63 

4 

35 

56 

1816.5 

1779 

1838 

1811.167 

- 794.833 

32 

8 

63 

5 

34 

56 

1778.5 

1735 

1787.5 

1767 

- 877.833 

33 

63 

0 

63 

0 

0 

3778 

3734 

3786.5 

3766.167 

- 960.333 

34 

63 

0 

63 

0 

0 

3568.5 

- 6356.5 

-6339 

- 3042.33 

- 971.167 

35 

59 

24 

41 

2 

47 

1332 

-8690 

-8663 

- 5340.33 

- 976.833 

36 

58 

42 

45 

3 

43 

479 

- 9566.5 

484 

- 2867.83 

- 983.667 

37 

58 

63 

50 

3 

37 

-525 

- 10570.5 

- 526.5 

-3874 

-987 

38 

58 

63 

50 

3 

37 

-534 

-10575 

-526 

- 3878.33 

- 1044.33 

39 

59 

0 

36 

2 

54 

2467.5 

-7575 

2474 

- 877.833 

- 1062.5 

40 

59 

3 

36 

2 

53 

2387 

- 7661.5 

2393.5 

- 960.333 

- 1071.17 

41 

36 

48 

58 

61 

12 

-714 

- 10763.5 

- 707.5 

- 4061.67 

- 1113.17 
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42 

36 

48 

58 

61 

12 

-750 

-10781 

- 699.5 

- 4076.83 

- 1213.17 

43 

18 

2 

54 

63 

9 

2710 

2704 

2756 

2723.333 

- 1228.83 

44 

42 

59 

53 

53 

12 

- 970.5 

-10973 

- 916.5 

- 4286.67 

- 1263.5 

45 

42 

59 

53 

53 

12 

- 957.5 

-10985 

-894 

- 4278.83 

- 1328.5 

46 

63 

58 

0 

33 

0 

2314 

2304 

2356.5 

2324.833 

- 1375.5 

47 

37 

39 

39 

53 

26 

246 

- 9747.5 

- 9741.5 

- 6414.33 

- 1406.33 

48 

39 

63 

63 

63 

0 

- 1365.5 

-11400 

- 1332.5 

- 4699.33 

- 1413.33 

49 

2 

0 

0 

17 

63 

5877 

- 4139.5 

5902.5 

2546.667 

- 1441.33 

50 

38 

54 

59 

52 

20 

-1031 

-11116 

- 11096.5 

- 7747.83 

- 1457.17 

51 

38 

63 

59 

63 

6 

-1437 

-11488 

-1397 

-4774 

- 1461.83 

52 

32 

27 

32 

23 

39 

2338 

-7693 

2404 

- 983.667 

-1694 

53 

32 

27 

32 

23 

39 

2370 

- 7668.5 

2385 

- 971.167 

- 1934.5 

54 

25 

34 

57 

37 

7 

1959.5 

- 8049.5 

- 8036.5 

- 4708.83 

- 1947.33 

55 

39 

59 

60 

61 

27 

- 2271.5 

- 2290.5 

- 2242.5 

- 2268.17 

- 1958.83 

56 

37 

59 

58 

61 

0 

- 703.5 

- 728.5 

-698 

-710 

- 1959.17 

57 

24 

60 

54 

49 

13 

-25 

- 10039.5 

- 1.5 

- 3355.33 

-1986 

58 

52 

58 

63 

63 

13 

- 2459.5 

- 12478.5 

- 2445.5 

- 5794.5 

- 2253.33 

59 

52 

58 

63 

63 

13 

- 2439.5 

- 2446.5 

-2399 

- 2428.33 

- 2268.17 

60 

43 

59 

56 

58 

15 

- 1485.5 

-11555 

-1499 

- 4846.5 

-2411 

61 

33 

59 

62 

63 

11 

- 1393.5 

-11440 

- 1343.5 

- 4725.67 

- 2428.33 

62 

63 

60 

46 

0 

40 

-449 

- 10491.5 

-394 

- 3778.17 

-2761 

63 

36 

18 

38 

52 

9 

2368.5 

-7674 

2375 

- 976.833 

- 2867.83 

64 

4 

8 

63 

63 

11 

2534.5 

- 7485.5 

2566.5 

- 794.833 

- 3042.33 

65 

4 

8 

63 

63 

11 

2536.5 

-7473 

2586 

- 783.5 

- 3355.33 

66 

14 

9 

37 

50 

25 

3186.5 

-6801 

-6797 

- 3470.5 

- 3407.17 

67 

14 

9 

37 

50 

25 

3255.5 

- 6758.5 

3297 

- 68.6667 

- 3470.5 

68 

0 

0 

0 

0 

63 

6912.5 

- 3141.5 

- 3128.5 

214.1667 

- 3704.5 

69 

2 

0 

1 

13 

59 

6320.5 

- 3722.5 

- 3705.5 

- 369.167 

- 3726.83 

70 

63 

63 

63 

63 

0 

-2495 

- 12569.5 

- 12549.5 

- 9204.67 

- 3727.83 

71 

42 

38 

59 

63 

5 

-373 

-10394 

- 346.5 

- 3704.5 

- 3758.33 

72 

23 

22 

49 

59 

20 

1336 

- 8693.5 

1399.5 

-1986 

- 3778.17 

73 

23 

22 

49 

59 

20 

1376 

-8678 

1425.5 

- 1958.83 

-3874 

74 

35 

41 

26 

28 

42 

1364 

- 8631.5 

1425.5 

- 1947.33 

- 3878.33 

75 

19 

9 

29 

44 

7 

4641.5 

4619 

4674 

4644.833 

-3986 

76 

21 

17 

63 

63 

13 

1107.5 

- 8879.5 

- 8863.5 

- 5545.17 

- 3990.83 

77 

34 

30 

58 

63 

28 

- 639.5 

-10686 

- 632.5 

-3986 

- 4061.67 

78 

34 

30 

58 

63 

28 

-600 

- 634.5 

- 584.5 

- 606.333 

- 4076.83 

79 

12 

4 

53 

63 

0 

3474.5 

3468.5 

3510 

3484.333 

- 4278.83 

80 

16 

8 

54 

63 

5 

2679 

- 7329.5 

-7322 

- 3990.83 

- 4286.67 

81 

16 

25 

52 

51 

23 

1619 

-8380 

1679 

-1694 

- 4300.83 

82 

24 

1 

58 

63 

0 

2723 

- 7328.5 

2725 

- 626.833 

- 4699.33 

83 

4 

63 

2 

22 

63 

2275 

-7733 

2325 

- 1044.33 

- 4708.83 

84 

33 

34 

54 

43 

22 

780 

740.5 

793.5 

771.3333 

- 4725.67 

85 

7 

0 

56 

63 

0 

3704 

- 6331.5 

3723.5 

365.3333 

- 4756.5 

86 

63 

63 

48 

0 

50 

-1208 

-11235 

-11218 

-7887 

-4774 

87 

42 

39 

2 

36 

19 

3131.5 

-6915 

3130 

- 217.833 

- 4803.5 

88 

22 

19 

2 

30 

37 

4489 

- 5535.5 

4521 

1158.167 

- 4846.5 

89 

62 

61 

33 

52 

1 

-434 

-10451 

-390 

- 3758.33 

- 4962.17 
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90 

51 

49 

13 

38 

11 

1859.5 

- 8145.5 

1900.5 

- 1461.83 

- 5340.33 

91 

42 

41 

22 

35 

22 

1865 

- 8146.5 

1910 

- 1457.17 

- 5545.17 

92 

42 

41 

22 

35 

22 

1895 

- 8137.5 

1918.5 

- 1441.33 

- 5658.5 

93 

16 

16 

48 

16 

48 

2863 

- 7185.5 

2868.5 

- 484.667 

- 5725.17 

94 

63 

63 

0 

63 

0 

583.5 

-9460 

593.5 

-2761 

- 5743.5 

95 

62 

61 

2 

52 

1 

1077 

- 8944.5 

1107.5 

- 2253.33 

- 5794.5 

96 

41 

39 

2 

27 

1 

4459.5 

- 5545.5 

4499.5 

1137.833 

- 6414.33 

97 

20 

19 

1 

14 

0 

7233 

-2749 

- 2727.5 

585.5 

- 7747.83 

98 

62 

61 

33 

52 

32 

-2120 

-12040 

-12022 

- 8727.33 

-7887 

99 

62 

61 

33 

52 

32 

- 1951.5 

-1981 

-1945 

- 1959.17 

- 8727.33 

100 

61 

58 

27 

52 

3 

- 81.5 

- 10095.5 

- 44.5 

- 3407.17 

- 9204.67 


ave 

-2003 


56 




Experiment 2, with Strategy 0.3 


Trials 

Aircraft Number by Type 
A1 A2 A3 A4 A5 

Scenariol 

Fitness Value 
Scenario2 

Scenario3 

Total 

Fitness 

sorted 

1 

0 

0 

0 

0 

0 

10014 

- 24.5 

10025 

6671.5 

8313.667 

2 

63 

63 

63 

63 

63 

-16634 

-16027 

-16000 

- 16220.3 

8055.333 

3 

3 

0 

1 

25 

55 

-4384 

5871 

5919 

2468.667 

6865.167 

4 

61 

58 

3 

41 

1 

- 8475.5 

- 8171.5 

- 8150.5 

- 8265.83 

6671.5 

5 

8 

62 

5 

34 

56 

1703 

-8296 

- 8293.5 

- 4962.17 

5767.833 

6 

59 

6 

37 

2 

52 

- 7843.5 

2234.5 

2287 

- 1107.33 

5715.5 

7 

38 

59 

59 

61 

13 

-1422 

-11515 

- 1459.5 

- 4798.83 

5056.5 

8 

58 

60 

49 

3 

38 

-11314 

-10441 

- 344.5 

- 7366.5 

4646 

9 

20 

13 

55 

63 

10 

1816 

-8075 

1974 

- 1428.33 

3873.667 

10 

0 

0 

0 

7 

12 

9045 

- 976.5 

9078 

5715.5 

3771.167 

11 

3 

0 

1 

32 

63 

5078 

5034.5 

5057 

5056.5 

3758.167 

12 

14 

3 

10 

1 

22 

- 3085.5 

-2460 

-2451 

- 2665.5 

3274.5 

13 

63 

9 

47 

3 

63 

- 9368.5 

-9266 

- 9245.5 

- 9293.33 

2602.833 

14 

2 

4 

6 

27 

1 

8044.5 

8034 

8087.5 

8055.333 

2468.667 

15 

22 

17 

61 

63 

11 

-8772 

1273 

1288.5 

- 2070.17 

1961.167 

16 

13 

20 

20 

20 

4 

- 4026.5 

- 3881.5 

6172.5 

- 578.5 

1392.167 

17 

25 

39 

39 

41 

9 

-8564 

-7686 

- 7673.5 

- 7974.5 

1273 

18 

51 

61 

61 

62 

38 

-3784 

- 13694.5 

- 13675.5 

- 10384.7 

1218.833 

19 

3 

21 

2 

11 

19 

7163.5 

7257.5 

-2800 

3873.667 

1151.833 

20 

5 

41 

3 

23 

37 

-5572 

- 5468.5 

- 5446.5 

- 5495.67 

1114.5 

21 

36 

63 

34 

49 

60 

- 12096.5 

-12106 

-12096 

- 12099.5 

993.6667 

22 

11 

10 

13 

1 

0 

8253.5 

8323 

8364.5 

8313.667 

744.3333 

23 

43 

42 

45 

33 

32 

364.5 

- 9788.5 

- 9771.5 

- 6398.5 

726.6667 

24 

26 

1 

11 

1 

12 

-2700 

- 2577.5 

7510.5 

744.3333 

704.1667 

25 

63 

61 

60 

4 

50 

- 1839.5 

-11932 

-11911 

- 8560.83 

654.8333 

26 

19 

16 

0 

16 

0 

-2734 

-2565 

7479 

726.6667 

376.5 

27 

19 

16 

0 

16 

0 

-3464 

-2590 

-2568 

-2874 

118.3333 

28 

21 

21 

21 

21 

21 

4720.5 

-5293 

4749 

1392.167 

- 46.5 

29 

42 

42 

42 

42 

42 

- 538.5 

-10524 

-10503 

- 7188.5 

- 413.667 

30 

22 

2 

13 

17 

54 

4660 

4616.5 

4661.5 

4646 

- 578.5 

31 

40 

4 

25 

10 

53 

3468.5 

-6585 

3471.5 

118.3333 

- 645.5 

32 

63 

6 

40 

0 

52 

1971.5 

-8050 

1997.5 

- 1360.33 

- 655.5 

33 

61 

6 

38 

1 

52 

2134 

-7916 

2138 

- 1214.67 

-719 

34 

61 

6 

38 

1 

52 

2134 

-7916 

2139.5 

- 1214.17 

- 736.833 

35 

5 

3 

21 

33 

47 

4625.5 

-5435 

-5422 

- 2077.17 

- 796.167 

36 

22 

16 

63 

63 

2 

1816 

-8265 

- 8249.5 

- 4899.5 

- 917.333 

37 

9 

12 

7 

40 

51 

4030 

- 5987.5 

4070 

704.1667 

- 1011.67 

38 

0 

0 

0 

10 

59 

6633 

- 3434.5 

6625 

3274.5 

- 1012.83 

39 

44 

63 

63 

63 

9 

- 2034.5 

- 12071.5 

- 12054.5 

- 8720.17 

- 1035.83 

40 

3 

22 

2 

27 

55 

4543.5 

-5492 

4605 

1218.833 

- 1107.33 

41 

3 

22 

2 

27 

55 

4623 

-5426 

4622 

1273 

- 1111.5 
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42 

8 

63 

6 

36 

56 

1661.5 

- 8420.5 

-8399 

- 5052.67 

- 1214.17 

43 

5 

6 

1 

27 

47 

5779.5 

5733.5 

5790.5 

5767.833 

- 1214.67 

44 

5 

6 

1 

27 

47 

5791.5 

- 4282.5 

-4261 

- 917.333 

- 1263.33 

45 

34 

38 

32 

34 

24 

2019 

-8063 

- 8046.5 

- 4696.83 

- 1282.17 

46 

21 

25 

2 

16 

47 

4449 

-5576 

4470.5 

1114.5 

- 1282.83 

47 

21 

25 

2 

16 

47 

4493 

-5549 

4511.5 

1151.833 

- 1308.33 

48 

63 

63 

50 

0 

30 

-172 

-10262 

-10248 

-6894 

- 1347.67 

49 

6 

4 

2 

29 

40 

5944 

-4078 

- 4076.5 

- 736.833 

- 1360.33 

50 

0 

0 

0 

21 

63 

-4200 

- 4188.5 

-4166 

- 4184.83 

- 1370.17 

51 

0 

0 

0 

21 

63 

5919 

-4162 

- 4145.5 

- 796.167 

-1379 

52 

63 

62 

4 

45 

0 

1254 

- 8742.5 

1300.5 

- 2062.67 

-1385 

53 

0 

0 

0 

9 

55 

6868.5 

6837 

6890 

6865.167 

- 1420.33 

54 

48 

8 

42 

5 

49 

2466 

- 7575.5 

- 7553.5 

-4221 

- 1428.33 

55 

63 

4 

32 

0 

55 

2333.5 

-7714 

2342 

- 1012.83 

- 1551.33 

56 

9 

15 

60 

63 

7 

2337 

- 7713.5 

2341.5 

- 1011.67 

- 2047.67 

57 

9 

15 

60 

63 

7 

2287.5 

- 7724.5 

2329.5 

- 1035.83 

- 2061.67 

58 

63 

1 

30 

0 

55 

2613 

2569.5 

2626 

2602.833 

- 2062.67 

59 

30 

63 

63 

63 

10 

-1413 

- 11461.5 

- 1406.5 

- 4760.33 

- 2070.17 

60 

30 

63 

63 

63 

10 

-1455 

- 11491.5 

- 1393.5 

-4780 

- 2077.17 

61 

42 

25 

26 

13 

53 

2089 

- 7965.5 

2086.5 

- 1263.33 

- 2084.83 

62 

25 

43 

16 

23 

55 

1966.5 

1929 

1988 

1961.167 

- 2665.5 

63 

0 

63 

0 

63 

63 

633 

640 

691.5 

654.8333 

- 2804.5 

64 

63 

0 

63 

0 

0 

3784 

3743.5 

3786 

3771.167 

-2874 

65 

61 

3 

50 

1 

26 

2818.5 

- 7106.5 

-7089 

- 3792.33 

- 2941.33 

66 

53 

18 

36 

6 

43 

2182 

-7840 

-7813 

- 4490.33 

- 2941.67 

67 

63 

0 

38 

0 

61 

1926.5 

-8124 

1936.5 

- 1420.33 

- 3412.83 

68 

26 

44 

31 

36 

23 

2042 

- 8013.5 

2046.5 

- 1308.33 

- 3727.67 

69 

59 

23 

43 

2 

48 

1249 

-8779 

1275.5 

- 2084.83 

- 3792.33 

70 

59 

0 

31 

2 

56 

2630.5 

- 7416.5 

2629 

-719 

- 3887.17 

71 

58 

63 

55 

3 

34 

- 611.5 

-10659 

-605 

- 3958.5 

- 3888.5 

72 

60 

23 

26 

15 

35 

2060.5 

- 7979.5 

2072.5 

- 1282.17 

- 3958.5 

73 

60 

41 

14 

28 

18 

1969 

-8079 

1973 

-1379 

- 4184.83 

74 

61 

59 

2 

42 

0 

1771 

- 8237.5 

1812.5 

- 1551.33 

-4221 

75 

61 

59 

2 

42 

0 

1753 

- 8246.5 

-8244 

- 4912.5 

- 4490.33 

76 

59 

3 

39 

0 

55 

2153 

-7846 

-7849 

-4514 

-4514 

77 

59 

3 

39 

0 

55 

2237 

- 7813.5 

2242 

- 1111.5 

- 4609.33 

78 

60 

20 

39 

28 

55 

-65 

- 10114.5 

-59 

- 3412.83 

- 4696.83 

79 

60 

20 

39 

28 

55 

- 28.5 

-79 

-32 

- 46.5 

- 4720.67 

80 

58 

0 

35 

0 

49 

2930.5 

- 7115.5 

2944 

- 413.667 

- 4760.33 

81 

32 

44 

58 

62 

12 

- 402.5 

-10441 

- 339.5 

- 3727.67 

-4780 

82 

63 

63 

63 

0 

63 

-2597 

-12641 

- 2546.5 

- 5928.17 

- 4798.83 

83 

51 

61 

61 

31 

38 

- 2056.5 

-2068 

- 2018.5 

- 2047.67 

- 4899.5 

84 

51 

61 

61 

31 

38 

-2065 

- 2085.5 

- 2034.5 

- 2061.67 

- 4912.5 

85 

10 

7 

28 

63 

5 

- 5684.5 

4324 

4341.5 

993.6667 

- 4962.17 

86 

16 

29 

38 

53 

25 

1963.5 

- 8068.5 

-8057 

- 4720.67 

- 5052.67 

87 

16 

29 

38 

53 

25 

1982 

-8075 

1982.5 

- 1370.17 

- 5495.67 

88 

0 

63 

0 

0 

63 

3759.5 

3728 

3787 

3758.167 

- 5928.17 

89 

4 

63 

3 

17 

60 

2692.5 

- 7343.5 

2714.5 

- 645.5 

- 6398.5 
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90 

4 

63 

3 

17 

60 

2694 

- 7357.5 

2697 

- 655.5 

-6894 

91 

63 

0 

63 

63 

0 

527 

- 9495.5 

555 

- 2804.5 

- 7188.5 

92 

24 

19 

47 

53 

17 

1986 

- 8043.5 

2014.5 

- 1347.67 

- 7366.5 

93 

28 

26 

40 

42 

25 

1952.5 

- 8080.5 

1973 

-1385 

- 7974.5 

94 

44 

52 

8 

0 

55 

2059 

-7980 

2072.5 

- 1282.83 

- 8265.83 

95 

38 

42 

20 

16 

43 

2087 

-7965 

-7950 

- 4609.33 

- 8560.83 

96 

0 

0 

63 

63 

0 

3730.5 

-6328 

3727 

376.5 

- 8720.17 

97 

10 

7 

59 

63 

5 

2793 

-7237 

- 7217.5 

- 3887.17 

- 9293.33 

98 

10 

7 

59 

63 

5 

2771.5 

-7227 

-7210 

- 3888.5 

- 10384.7 

99 

45 

44 

51 

23 

29 

397.5 

-9638 

415.5 

- 2941.67 

- 12099.5 

100 

45 

44 

51 

23 

29 

397 

- 9635.5 

414.5 

- 2941.33 

- 16220.3 


ave 

- 1820.17 
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Experiment 3, with Strategy 0.5 


Trials 

Aircraft Number by Type 
A1 A2 A3 A4 A5 

Scenariol 

Fitness Value 
Scenario2 

Scenario3 

Total 

Fitness 

sorted 

1 

0 

0 

0 

0 

0 

10014 

- 24.5 

10025 

6671.5 

6671.5 

2 

0 

0 

0 

0 

0 

- 267.5 

-277 

-250 

- 264.833 

6046 

3 

3 

0 

1 

25 

55 

5884.5 

5871 

5919 

5891.5 

5990.5 

4 

61 

58 

3 

41 

1 

1911.5 

- 8171.5 

- 8150.5 

- 4803.5 

5891.5 

5 

8 

62 

5 

34 

56 

1703 

-8296 

- 8293.5 

- 4962.17 

3514 

6 

38 

59 

59 

61 

13 

-1465 

-11515 

- 1459.5 

- 4813.17 

3489.333 

7 

58 

60 

49 

3 

38 

-395 

-10441 

- 344.5 

- 3726.83 

3340.333 

8 

20 

13 

55 

63 

10 

1967.5 

-8075 

1974 

- 1377.83 

3327.5 

9 

20 

13 

55 

63 

10 

1945 

- 8076.5 

1978 

- 1384.5 

3182.5 

10 

0 

0 

0 

15 

53 

6690.5 

- 3380.5 

6672.5 

3327.5 

2794.833 

11 

10 

15 

10 

27 

45 

4712 

- 5323.5 

- 5306.5 

- 1972.67 

2584.333 

12 

0 

0 

0 

23 

63 

5740 

- 4292.5 

5763.5 

2403.667 

2403.667 

13 

0 

0 

0 

23 

63 

5819.5 

-4263 

- 4245.5 

- 896.333 

2314.5 

14 

39 

47 

41 

34 

22 

844 

- 9180.5 

865.5 

- 2490.33 

2271 

15 

40 

40 

33 

10 

44 

1687 

-8347 

1704 

-1652 

2223.333 

16 

61 

63 

51 

2 

37 

- 670.5 

-10720 

-676 

- 4022.17 

2093.833 

17 

61 

63 

51 

2 

37 

-685 

- 10724.5 

-676 

- 4028.5 

1795.833 

18 

0 

0 

0 

63 

63 

3825.5 

- 6260.5 

3794.5 

453.1667 

1403.5 

19 

22 

19 

2 

30 

37 

4624 

- 5463.5 

-5454 

- 2097.83 

1184.167 

20 

42 

39 

2 

36 

19 

3116 

-6901 

3160 

- 208.333 

1154.833 

21 

63 

63 

63 

63 

0 

-2611 

- 12635.5 

-2579 

- 5941.83 

1121.167 

22 

62 

61 

33 

52 

1 

-473 

-10494 

- 446.5 

- 3804.5 

1119 

23 

15 

20 

20 

37 

41 

3309.5 

- 6695.5 

3350.5 

- 11.8333 

602 

24 

26 

39 

40 

49 

27 

993.5 

- 9049.5 

1003.5 

- 2350.83 

453.1667 

25 

51 

61 

61 

62 

7 

- 2068.5 

-2091 

- 2031.5 

- 2063.67 

387 

26 

5 

21 

2 

28 

55 

4439.5 

- 5587.5 

4505 

1119 

121.6667 

27 

6 

41 

4 

31 

56 

3168.5 

-6876 

3171.5 

- 178.667 

- 11.8333 

28 

36 

63 

34 

49 

60 

-2028 

- 12095.5 

-2043 

- 5388.83 

-63 

29 

2 

0 

1 

24 

55 

5984 

5973 

6014.5 

5990.5 

- 178.667 

30 

4 

0 

1 

26 

55 

5811.5 

-4270 

- 4253.5 

-904 

- 208.333 

31 

28 

0 

9 

18 

54 

4660.5 

-5423 

-5397 

- 2053.17 

- 264.833 

32 

0 

0 

0 

32 

56 

5666 

- 4386.5 

5664 

2314.5 

- 369.167 

33 

63 

6 

45 

0 

51 

1870 

-8217 

-8199 

- 4848.67 

- 412.167 

34 

9 

4 

19 

38 

40 

4533 

- 5516.5 

4536 

1184.167 

- 597.333 

35 

14 

9 

37 

50 

25 

3312.5 

- 6741.5 

- 6728.5 

- 3385.83 

- 611.167 

36 

42 

38 

59 

63 

5 

- 344.5 

- 10358.5 

-303 

- 3668.67 

- 670.5 

37 

0 

0 

0 

0 

63 

6836 

- 3193.5 

6899.5 

3514 

- 712.167 

38 

2 

0 

1 

13 

59 

6320.5 

- 3722.5 

- 3705.5 

- 369.167 

- 736.333 

39 

53 

53 

53 

53 

53 

-3145 

- 13219.5 

- 13199.5 

- 9854.67 

- 816.333 

40 

42 

42 

42 

42 

42 

-420 

-429 

- 387.5 

- 412.167 

- 896.333 

41 

16 

16 

16 

16 

16 

6060 

6016.5 

6061.5 

6046 

-904 
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42 

61 

62 

58 

43 

55 

-3995 

-13977 

- 13959.5 

- 10643.8 

- 1126.67 

43 

60 

61 

54 

23 

46 

-2119 

-2127 

-2086 

- 2110.67 

-1133 

44 

60 

61 

54 

23 

46 

-2155 

-2178 

-2132 

-2155 

- 1168.83 

45 

58 

60 

49 

2 

37 

- 286.5 

- 10323.5 

-276 

- 3628.67 

- 1275.33 

46 

58 

60 

49 

2 

37 

-285 

- 10324.5 

-276 

- 3628.5 

- 1377.83 

47 

62 

60 

23 

48 

22 

- 679.5 

-691 

-641 

- 670.5 

- 1384.5 

48 

61 

58 

2 

41 

0 

1910.5 

- 8131.5 

1954.5 

- 1422.17 

- 1422.17 

49 

60 

62 

62 

63 

54 

- 5097.5 

-15096 

-15098 

- 11763.8 

- 1527.5 

50 

35 

58 

58 

61 

4 

- 719.5 

- 729.5 

- 687.5 

- 712.167 

-1652 

51 

35 

58 

58 

61 

4 

- 817.5 

-10843 

- 791.5 

- 4150.67 

- 1869.83 

52 

26 

62 

24 

44 

58 

- 615.5 

-632 

-586 

- 611.167 

- 1909.67 

53 

4 

31 

3 

17 

28 

5933 

-4150 

5970 

2584.333 

- 1934.83 

54 

4 

31 

3 

17 

28 

5861.5 

-4163 

- 4147.5 

- 816.333 

- 1972.67 

55 

63 

43 

54 

49 

59 

- 3438.5 

-13424 

-13403 

- 10088.5 

- 2029.5 

56 

59 

0 

28 

0 

48 

3333.5 

3323 

3364.5 

3340.333 

- 2038.17 

57 

59 

0 

28 

0 

48 

3279.5 

- 6762.5 

3294 

-63 

- 2053.17 

58 

17 

3 

55 

63 

0 

3174 

3163 

3210.5 

3182.5 

- 2063.67 

59 

41 

44 

33 

21 

35 

1261 

- 8729.5 

-8717 

- 5395.17 

- 2097.83 

60 

41 

44 

33 

21 

35 

1309.5 

- 8720.5 

1322.5 

- 2029.5 

- 2110.67 

61 

63 

63 

50 

0 

41 

- 824.5 

- 10868.5 

-811 

-4168 

-2155 

62 

63 

63 

50 

0 

41 

-830 

-10874 

- 824.5 

- 4176.17 

-2307 

63 

31 

30 

42 

30 

41 

1296.5 

- 8734.5 

1323.5 

- 2038.17 

- 2350.83 

64 

62 

60 

0 

43 

0 

1792 

1765.5 

1830 

1795.833 

- 2474.67 

65 

62 

60 

0 

43 

0 

1703 

-8302 

- 8289.5 

- 4962.83 

- 2490.33 

66 

29 

21 

29 

20 

32 

3488 

3463.5 

3516.5 

3489.333 

- 2998.83 

67 

29 

21 

29 

20 

32 

3450.5 

-6575 

3489.5 

121.6667 

- 3385.83 

68 

25 

40 

25 

32 

34 

2210.5 

2203.5 

2256 

2223.333 

- 3628.5 

69 

25 

40 

25 

32 

34 

2220 

- 7826.5 

2226.5 

- 1126.67 

- 3628.67 

70 

1 

63 

0 

34 

58 

2210 

- 7832.5 

2223.5 

-1133 

- 3642.67 

71 

1 

63 

0 

34 

58 

2278 

2241 

2294 

2271 

- 3655.33 

72 

19 

29 

22 

17 

29 

4203 

-5819 

-5808 

- 2474.67 

- 3668.67 

73 

41 

23 

34 

22 

39 

2100 

2064 

2117.5 

2093.833 

- 3726.83 

74 

41 

23 

34 

22 

39 

2069 

-7978 

2083 

- 1275.33 

- 3777.33 

75 

50 

15 

35 

12 

45 

2181.5 

- 7868.5 

2180.5 

- 1168.83 

- 3777.67 

76 

61 

3 

50 

1 

58 

1393.5 

1380.5 

1436.5 

1403.5 

- 3804.5 

77 

0 

63 

0 

63 

0 

3739.5 

- 6315.5 

3737 

387 

- 4022.17 

78 

0 

63 

0 

63 

0 

3663.5 

- 6337.5 

- 6322.5 

- 2998.83 

- 4028.5 

79 

29 

23 

38 

36 

22 

2596.5 

- 7429.5 

2624 

- 736.333 

- 4150.67 

80 

35 

41 

26 

28 

42 

1396.5 

- 8631.5 

1430.5 

- 1934.83 

-4168 

81 

17 

4 

61 

63 

0 

2791.5 

2769 

2824 

2794.833 

- 4176.17 

82 

17 

4 

61 

63 

0 

2714 

- 7281.5 

2775.5 

- 597.333 

- 4488.67 

83 

58 

61 

63 

0 

49 

-1544 

- 11581.5 

- 1533.5 

- 4886.33 

- 4803.5 

84 

61 

57 

0 

53 

0 

1480.5 

-8565 

1475 

- 1869.83 

- 4813.17 

85 

61 

57 

0 

53 

0 

1411 

- 8591.5 

1451.5 

- 1909.67 

- 4823.5 

86 

45 

59 

56 

42 

21 

- 1151.5 

-11175 

- 1139.5 

- 4488.67 

- 4828.33 

87 

37 

59 

59 

63 

12 

-1499 

-11526 

- 1445.5 

- 4823.5 

- 4848.67 

88 

37 

59 

59 

63 

12 

-1494 

- 11541.5 

- 1449.5 

- 4828.33 

- 4886.33 

89 

38 

59 

59 

62 

13 

-1542 

-1546 

- 1494.5 

- 1527.5 

- 4962.17 
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90 

59 

60 

49 

2 

39 

- 433.5 

-10474 

- 425.5 

- 3777.67 

- 4962.83 

91 

59 

60 

49 

2 

39 

-432 

- 10474.5 

- 425.5 

- 3777.33 

- 5142.33 

92 

25 

61 

20 

24 

50 

1030.5 

-9019 

1067.5 

-2307 

- 5388.83 

93 

0 

63 

0 

63 

63 

606.5 

576 

623.5 

602 

- 5395.17 

94 

4 

63 

3 

49 

60 

1133 

1140 

1191.5 

1154.833 

- 5941.83 

95 

4 

63 

3 

49 

60 

1134 

1093.5 

1136 

1121.167 

- 9854.67 

96 

61 

30 

56 

2 

19 

1468.5 

- 8456.5 

-8439 

- 5142.33 

- 10088.5 

97 

39 

40 

33 

2 

25 

3006.5 

-6993 

- 6979.5 

- 3655.33 

- 10643.8 

98 

39 

40 

33 

2 

25 

3031 

-6990 

-6969 

- 3642.67 

- 11763.8 


average 

- 1548.71 
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Experiment 4, with Strategy 0.8 


Aircraft Number by Type Fitness Value Total 


Trials 

A1 

A2 

A3 

A4 

A5 

Scenariol 

Scenario2 

Scenario3 

Fitness 

sorted 

1 

0 

0 

0 

0 

0 

- 191.5 

- 24.5 

10025 

3269.667 

6893.667 

2 

0 

0 

0 

0 

0 

- 267.5 

-277 

-250 

- 264.833 

6140.667 

3 

3 

0 

1 

25 

55 

5884.5 

5871 

5919 

5891.5 

6063.5 

4 

61 

58 

3 

41 

1 

1911.5 

- 8171.5 

- 8150.5 

- 4803.5 

5990.5 

5 

8 

62 

5 

34 

56 

1703 

-8296 

- 8293.5 

- 4962.17 

5891.5 

6 

59 

6 

37 

2 

52 

2278 

2234.5 

2287 

2266.5 

4818 

7 

38 

59 

59 

61 

13 

-1465 

-11515 

- 1459.5 

- 4813.17 

4199.833 

8 

58 

60 

49 

3 

38 

-395 

-10441 

- 344.5 

- 3726.83 

3770.833 

9 

20 

13 

55 

63 

10 

1967.5 

-8075 

1974 

- 1377.83 

3522 

10 

20 

13 

55 

63 

10 

1945 

- 8076.5 

1978 

- 1384.5 

3269.667 

11 

0 

0 

0 

31 

56 

5745 

- 4327.5 

-4305 

- 962.5 

2497.833 

12 

63 

7 

43 

0 

51 

1915 

- 8171.5 

-8149 

- 4801.83 

2266.5 

13 

23 

21 

22 

38 

58 

1933 

- 8116.5 

1936.5 

- 1415.67 

1149.333 

14 

43 

42 

42 

50 

60 

-1786 

-1810 

- 1751.5 

- 1782.5 

1119 

15 

43 

42 

42 

50 

60 

-1766 

-1778 

- 1732.5 

- 1758.83 

993.6667 

16 

2 

0 

1 

13 

28 

- 2208.5 

-2196 

- 2174.5 

-2193 

602 

17 

22 

21 

22 

30 

40 

3292.5 

-6750 

3315 

- 47.5 

453.1667 

18 

63 

63 

63 

63 

0 

- 2576.5 

- 12605.5 

-2556 

- 5912.67 

374.6667 

19 

48 

48 

48 

48 

16 

-423 

-10444 

- 396.5 

- 3754.5 

372.1667 

20 

0 

0 

0 

0 

63 

6846.5 

-3188 

6907.5 

3522 

371.6667 

21 

2 

0 

1 

13 

59 

6320.5 

- 3722.5 

- 3705.5 

- 369.167 

- 11.8333 

22 

21 

20 

17 

18 

49 

3855 

- 6219.5 

- 6199.5 

- 2854.67 

- 47.5 

23 

40 

40 

33 

10 

44 

1687 

-8347 

1704 

-1652 

- 178.667 

24 

40 

40 

33 

10 

44 

1679.5 

-8370 

1674 

- 1672.17 

- 208.333 

25 

59 

62 

50 

2 

38 

-535 

- 10574.5 

-526 

- 3878.5 

- 241.833 

26 

0 

0 

0 

63 

63 

3716.5 

- 6324.5 

3724.5 

372.1667 

- 264.833 

27 

0 

0 

0 

63 

63 

3825.5 

- 6260.5 

3794.5 

453.1667 

- 369.167 

28 

22 

19 

2 

30 

37 

4624 

- 5463.5 

-5454 

- 2097.83 

-409 

29 

42 

39 

2 

36 

19 

3116 

-6901 

3160 

- 208.333 

- 421.833 

30 

42 

39 

2 

36 

19 

3089 

- 6935.5 

3121 

- 241.833 

-428 

31 

15 

20 

20 

37 

41 

3309.5 

- 6695.5 

3350.5 

- 11.8333 

- 434.333 

32 

26 

39 

40 

49 

27 

993.5 

- 9049.5 

1003.5 

- 2350.83 

- 565.667 

33 

51 

61 

61 

62 

7 

- 2068.5 

-2091 

- 2031.5 

- 2063.67 

- 640.833 

34 

5 

21 

2 

28 

55 

4439.5 

- 5587.5 

4505 

1119 

- 681.833 

35 

6 

41 

4 

31 

56 

3168.5 

-6876 

3171.5 

- 178.667 

- 708.5 

36 

36 

63 

34 

49 

60 

-2028 

- 12095.5 

-2043 

- 5388.83 

- 772.5 

37 

2 

0 

1 

24 

55 

5984 

5973 

6014.5 

5990.5 

- 903.167 

38 

4 

0 

1 

26 

55 

5811.5 

-4270 

- 4253.5 

-904 

-904 

39 

4 

0 

1 

26 

55 

5810.5 

-4273 

-4247 

- 903.167 

- 962.5 

40 

0 

0 

0 

13 

63 

6283 

-3790 

-3777 

-428 

- 1097.33 

41 

28 

13 

63 

63 

0 

1761 

- 8318.5 

-8297 

- 4951.5 

- 1125.67 
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42 

28 

13 

63 

63 

0 

1627 

-8391 

1668 

- 1698.67 

- 1212.83 

43 

62 

44 

54 

43 

59 

- 3032.5 

-3067 

- 3030.5 

- 3043.33 

- 1220.5 

44 

30 

3 

19 

1 

26 

6133.5 

6123.5 

6165 

6140.667 

- 1255.67 

45 

30 

3 

19 

1 

26 

6010 

-3975 

- 3957.5 

- 640.833 

- 1262.83 

46 

41 

23 

34 

22 

39 

2081.5 

- 7968.5 

2080.5 

- 1268.83 

- 1268.83 

47 

0 

63 

0 

63 

0 

3719 

-6328 

3733 

374.6667 

- 1372.17 

48 

16 

48 

16 

48 

16 

2763.5 

- 7237.5 

- 7222.5 

- 3898.83 

- 1377.83 

49 

61 

4 

37 

0 

53 

2213.5 

- 7779.5 

2274 

- 1097.33 

- 1384.5 

50 

60 

5 

37 

1 

53 

2196.5 

- 7814.5 

2241 

- 1125.67 

-1398 

51 

59 

24 

41 

2 

47 

- 8839.5 

- 8664.5 

1391 

-5371 

- 1415.67 

52 

58 

42 

45 

3 

43 

485 

- 9566.5 

484 

- 2865.83 

- 1430.33 

53 

58 

63 

50 

3 

37 

-521 

- 10570.5 

- 526.5 

- 3872.67 

- 1512.67 

54 

58 

62 

49 

3 

38 

- 476.5 

-10525 

-476 

- 3825.83 

-1652 

55 

63 

0 

0 

0 

63 

3716 

-6325 

3724 

371.6667 

- 1672.17 

56 

61 

3 

19 

1 

58 

2917.5 

- 7096.5 

2952 

-409 

- 1698.67 

57 

60 

23 

26 

15 

35 

2092.5 

-7956 

2096.5 

- 1255.67 

- 1758.83 

58 

60 

41 

14 

28 

18 

1989.5 

-8079 

1973 

- 1372.17 

- 1782.5 

59 

61 

59 

2 

42 

0 

-8384 

- 8237.5 

1812.5 

- 4936.33 

- 1928.17 

60 

61 

59 

3 

41 

1 

1703 

- 8296.5 

-8294 

- 4962.5 

- 1938.5 

61 

59 

3 

39 

0 

55 

- 8702.5 

-7846 

-7849 

- 8132.5 

- 2063.67 

62 

59 

5 

38 

1 

53 

-7849 

- 7813.5 

2242 

- 4473.5 

- 2097.83 

63 

57 

19 

45 

15 

45 

- 9952.5 

- 9064.5 

991 

- 6008.67 

- 2152.67 

64 

61 

0 

29 

0 

59 

2585 

-7477 

2574.5 

- 772.5 

-2193 

65 

36 

63 

63 

63 

6 

- 11748.5 

- 11558.5 

-1500 

-8269 

- 2320.67 

66 

42 

25 

26 

13 

53 

2100 

-7993 

2104.5 

- 1262.83 

- 2350.83 

67 

25 

43 

16 

23 

55 

1888 

- 8115.5 

1936.5 

- 1430.33 

- 2854.67 

68 

0 

63 

0 

63 

63 

589 

579 

638 

602 

- 2865.83 

69 

4 

63 

3 

49 

60 

1116.5 

1140 

1191.5 

1149.333 

- 2911.17 

70 

63 

0 

63 

0 

0 

3783 

3743.5 

3786 

3770.833 

- 3043.33 

71 

61 

3 

50 

1 

26 

3034 

- 7106.5 

-7089 

- 3720.5 

- 3720.5 

72 

39 

4 

25 

1 

35 

4668.5 

-5240 

-5213 

- 1928.17 

- 3726.83 

73 

20 

2 

12 

1 

17 

-3514 

-2615 

- 2604.5 

- 2911.17 

- 3754.5 

74 

63 

6 

40 

2 

56 

1643.5 

-8376 

-8355 

- 5029.17 

- 3825.83 

75 

61 

6 

38 

2 

54 

1879.5 

-8066 

1992.5 

-1398 

- 3872.67 

76 

58 

8 

43 

16 

39 

1838.5 

- 8215.5 

1839 

- 1512.67 

- 3878.5 

77 

60 

4 

31 

0 

63 

2137 

- 7929.5 

2131 

- 1220.5 

- 3898.83 

78 

19 

15 

61 

63 

0 

2125 

- 7911.5 

2148 

- 1212.83 

- 4076.83 

79 

53 

53 

53 

53 

53 

- 14152.5 

- 13286.5 

-3229 

- 10222.7 

- 4473.5 

80 

42 

42 

42 

42 

42 

-11414 

-429 

- 387.5 

- 4076.83 

- 4801.83 

81 

16 

16 

16 

16 

16 

6112.5 

6016.5 

6061.5 

6063.5 

- 4803.5 

82 

61 

62 

58 

43 

55 

-3890 

-13977 

- 13959.5 

- 10608.8 

- 4813.17 

83 

60 

61 

54 

23 

46 

-2245 

-2127 

-2086 

- 2152.67 

- 4936.33 

84 

29 

30 

25 

2 

19 

4864 

4772 

4818 

4818 

- 4951.5 

85 

62 

61 

43 

56 

42 

-3155 

- 13238.5 

- 13219.5 

-9871 

- 4962.17 

86 

62 

60 

23 

48 

22 

- 793.5 

-691 

-641 

- 708.5 

- 4962.5 

87 

31 

29 

2 

21 

1 

5870.5 

- 4231.5 

5854.5 

2497.833 

- 5029.17 

88 

55 

62 

62 

62 

46 

- 4339.5 

- 14395.5 

- 14380.5 

- 11038.5 

-5371 

89 

46 

60 

60 

62 

30 

- 13076.5 

- 2831.5 

-2789 

- 6232.33 

- 5388.83 
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90 

19 

30 

30 

31 

7 

4225.5 

4162 

4212 

4199.833 

- 5912.67 

91 

60 

63 

42 

56 

61 

- 4046.5 

-14140 

-14119 

- 10768.5 

- 6008.67 

92 

60 

63 

42 

56 

61 

-14284 

- 4027.5 

-3986 

- 7432.5 

- 6232.33 

93 

5 

62 

0 

27 

54 

2668.5 

- 7405.5 

2691.5 

- 681.833 

- 7432.5 

94 

21 

21 

21 

21 

21 

4711.5 

-5274 

-5253 

- 1938.5 

- 8132.5 

95 

34 

30 

58 

63 

28 

- 575.5 

- 581.5 

-540 

- 565.667 

-8269 

96 

0 

0 

0 

63 

0 

6900 

6865.5 

6915.5 

6893.667 

-9871 

97 

10 

7 

28 

63 

5 

- 5684.5 

4324 

4341.5 

993.6667 

- 10222.7 

98 

10 

7 

28 

63 

5 

4363.5 

- 5668.5 

-5657 

- 2320.67 

- 10608.8 

99 

25 

24 

28 

33 

32 

2913 

-7133 

2917 

- 434.333 

- 10768.5 

100 

25 

24 

28 

33 

32 

2915 

- 7119.5 

2939 

- 421.833 

- 11038.5 


average 

- 1837.59 
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Appendix 3: Test Experiment with 249 runs 


Aircraft Number by Type Fitness Value Total 


Trials 

A1 

A2 

A3 

A4 

A5 

Scenariol 

Scenario2 

Scenario3 

Fitness 

rank 

max sorted 

1 

0 

0 

0 

0 

0 

10014 

-24.5 

10025 

6671.5 

5 

8313.667 8313.667 

2 

63 

63 

63 

63 

63 

-16634 

-16027 

-16000 

-16220.3 

249 

8055.333 

3 

3 

0 

1 

25 

55 

-4384 

5871 

5919 

2468.667 

34 

Online 7593.333 

4 

61 

58 

3 

41 

1 

-8475.5 

-8171.5 

-8150.5 

-8265.83 

238 

-2993.81 6865.167 

5 

8 

62 

5 

34 

56 

1703 

-8296 

-8293.5 

-4962.17 

217 

-2143.31 6390.833 

6 

59 

6 

37 

2 

52 

-7843.5 

2234.5 

2287 

-1107.33 

110 

-1718.64 5987.667 

7 

38 

59 

59 

61 

13 

-1422 

-11515 

-1459.5 

-4798.83 

213 

-1841.26 5938.833 

8 

58 

60 

49 

3 

38 

-11314 

-10441 

-344.5 

-7366.5 

235 

-1786.97 5935.833 

9 

20 

13 

55 

63 

10 

1816 

-8075 

1974 

-1428.33 

129 

-1910.79 5801 

10 

0 

0 

0 

7 

12 

9045 

-976.5 

9078 

5715.5 

12 

-1887.14 5767.833 

11 

3 

0 

1 

32 

63 

5078 

5034.5 

5057 

5056.5 

16 

-1865.86 5715.5 

12 

14 

3 

10 

1 

22 

-3085.5 

-2460 

-2451 

-2665.5 

165 

-1733.33 5663.833 

13 

63 

9 

47 

3 

63 

-9368.5 

-9266 

-9245.5 

-9293.33 

243 

-1558.45 5418.167 

14 

2 

4 

6 

27 

1 

8044.5 

8034 

8087.5 

8055.333 

2 

-1438.93 5090.833 

15 

22 

17 

61 

63 

11 

-8772 

1273 

1288.5 

-2070.17 

145 

-1336.6 5056.5 

16 

13 

20 

20 

20 

4 

-4026.5 

-3881.5 

6172.5 

-578.5 

95 

4849 

17 

25 

39 

39 

41 

9 

-8564 

-7686 

-7673.5 

-7974.5 

236 

Offline 4839.333 

18 

51 

61 

61 

62 

38 

-3784 

-13694.5 

-13675.5 

-10384.7 

245 

8055.333 4668 

19 

3 

21 

2 

11 

19 

7163.5 

7257.5 

-2800 

3873.667 

24 

8184.5 4646 

20 

5 

41 

3 

23 

37 

-5572 

-5468.5 

-5446.5 

-5495.67 

224 

8227.556 4379.833 

21 

36 

63 

34 

49 

60 

-12096.5 

-12106 

-12096 

-12099.5 

247 

8249.083 4298.667 

22 

11 

10 

13 

1 

0 

8253.5 

8323 

8364.5 

8313.667 

1 

8262 4013.167 

23 

43 

42 

45 

33 

32 

364.5 

-9788.5 

-9771.5 

-6398.5 

231 

8270.611 3873.667 

24 

26 

1 

11 

1 

12 

-2700 

-2577.5 

7510.5 

744.3333 

59 

8276.762 3771.167 

25 

63 

61 

60 

4 

50 

-1839.5 

-11932 

-11911 

-8560.83 

240 

8281.375 3766.5 

26 

19 

16 

0 

16 

0 

-2734 

-2565 

7479 

726.6667 

61 

8284.963 3758.167 

27 

19 

16 

0 

16 

0 

-3464 

-2590 

-2568 

-2874 

172 

8287.833 3530 

28 

21 

21 

21 

21 

21 

4720.5 

-5293 

4749 

1392.167 

45 

8290.182 3484.333 

29 

42 

42 

42 

42 

42 

-538.5 

-10524 

-10503 

-7188.5 

234 

8292.139 3274.5 

30 

22 

2 

13 

17 

54 

4660 

4616.5 

4661.5 

4646 

20 

2843 

31 

40 

4 

25 

10 

53 

3468.5 

-6585 

3471.5 

118.3333 

83 

Best 2682 

32 

63 

6 

40 

0 

52 

1971.5 

-8050 

1997.5 

-1360.33 

124 

8055.333 2602.833 

33 

61 

6 

38 

1 

52 

2134 

-7916 

2138 

-1214.67 

113 

8313.667 2468.667 

34 

61 

6 

38 

1 

52 

2134 

-7916 

2139.5 

-1214.17 

112 

8313.667 2273.833 

35 

5 

3 

21 

33 

47 

4625.5 

-5435 

-5422 

-2077.17 

146 

8313.667 2109.5 

36 

22 

16 

63 

63 

2 

1816 

-8265 

-8249.5 

-4899.5 

214 

8313.667 2034.333 

37 

9 

12 

7 

40 

51 

4030 

-5987.5 

4070 

704.1667 

65 

8313.667 2000.833 

38 

0 

0 

0 

10 

59 

6633 

-3434.5 

6625 

3274.5 

30 

8313.667 1961.167 

39 

44 

63 

63 

63 

9 

-2034.5 

-12071.5 

-12054.5 

-8720.17 

241 

8313.667 1840.167 

40 

3 

22 

2 

27 

55 

4543.5 

-5492 

4605 

1218.833 

51 

8313.667 1795.167 

41 

3 

22 

2 

27 

55 

4623 

-5426 

4622 

1273 

49 

8313.667 1549.833 

42 

8 

63 

6 

36 

56 

1661.5 

-8420.5 

-8399 

-5052.67 

218 

8313.667 1546.833 
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43 

5 

6 

1 

27 

47 

5779.5 

5733.5 

5790.5 

5767.833 

11 

8313.667 1397.667 

44 

5 

6 

1 

27 

47 

5791.5 

-4282.5 

-4261 

-917.333 

105 

1392.167 

45 

34 

38 

32 

34 

24 

2019 

-8063 

-8046.5 

-4696.83 

209 

Average 1376.5 

46 

21 

25 

2 

16 

47 

4449 

-5576 

4470.5 

1114.5 

53 

-2447.9 1353.167 

47 

21 

25 

2 

16 

47 

4493 

-5549 

4511.5 

1151.833 

52 

-1153.02 1285.167 

48 

63 

63 

50 

0 

30 

-172 

-10262 

-10248 

-6894 

233 

-1101.29 1273 

49 

6 

4 

2 

29 

40 

5944 

-4078 

-4076.5 

-736.833 

100 

-2385.57 1259.667 

50 

0 

0 

0 

21 

63 

-4200 

-4188.5 

-4166 

-4184.83 

198 

-1655.94 1218.833 

51 

0 

0 

0 

21 

63 

5919 

-4162 

-4145.5 

-796.167 

102 

-2845.02 1151.833 

52 

63 

62 

4 

45 

0 

1254 

-8742.5 

1300.5 

-2062.67 

144 

-1292.4 1114.5 

53 

0 

0 

0 

9 

55 

6868.5 

6837 

6890 

6865.167 

4 

-1618.65 1084.667 

54 

48 

8 

42 

5 

49 

2466 

-7575.5 

-7553.5 

-4221 

199 

-643.595 993.6667 

55 

63 

4 

32 

0 

55 

2333.5 

-7714 

2342 

-1012.83 

107 

29.56349 958.8333 

56 

9 

15 

60 

63 

7 

2337 

-7713.5 

2341.5 

-1011.67 

106 

-406.968 800 

57 

9 

15 

60 

63 

7 

2287.5 

-7724.5 

2329.5 

-1035.83 

108 

-828.5 756.8333 

58 

63 

1 

30 

0 

55 

2613 

2569.5 

2626 

2602.833 

33 

744.3333 

59 

30 

63 

63 

63 

10 

-1413 

-11461.5 

-1406.5 

-4760.33 

211 

743 

60 

30 

63 

63 

63 

10 

-1455 

-11491.5 

-1393.5 

-4780 

212 

726.6667 

61 

42 

25 

26 

13 

53 

2089 

-7965.5 

2086.5 

-1263.33 

115 

713.8333 

62 

25 

43 

16 

23 

55 

1966.5 

1929 

1988 

1961.167 

39 

708.1667 

63 

0 

63 

0 

63 

63 

633 

640 

691.5 

654.8333 

67 

706.8333 

64 

63 

0 

63 

0 

0 

3784 

3743.5 

3786 

3771.167 

25 

704.1667 

65 

61 

3 

50 

1 

26 

2818.5 

-7106.5 

-7089 

-3792.33 

185 

659.8333 

66 

53 

18 

36 

6 

43 

2182 

-7840 

-7813 

-4490.33 

205 

654.8333 

67 

63 

0 

38 

0 

61 

1926.5 

-8124 

1936.5 

-1420.33 

128 

651.3333 

68 

26 

44 

31 

36 

23 

2042 

-8013.5 

2046.5 

-1308.33 

120 

603.8333 

69 

59 

23 

43 

2 

48 

1249 

-8779 

1275.5 

-2084.83 

147 

562.6667 

70 

59 

0 

31 

2 

56 

2630.5 

-7416.5 

2629 

-719 

99 

482 

71 

58 

63 

55 

3 

34 

-611.5 

-10659 

-605 

-3958.5 

191 

457 

72 

60 

23 

26 

15 

35 

2060.5 

-7979.5 

2072.5 

-1282.17 

117 

425.1667 

73 

60 

41 

14 

28 

18 

1969 

-8079 

1973 

-1379 

126 

406.6667 

74 

61 

59 

2 

42 

0 

1771 

-8237.5 

1812.5 

-1551.33 

133 

376.5 

75 

61 

59 

2 

42 

0 

1753 

-8246.5 

-8244 

-4912.5 

215 

369.5 

76 

59 

3 

39 

0 

55 

2153 

-7846 

-7849 

-4514 

206 

343.8333 

77 

59 

3 

39 

0 

55 

2237 

-7813.5 

2242 

-1111.5 

111 

341.5 

78 

60 

20 

39 

28 

55 

-65 

-10114.5 

-59 

-3412.83 

181 

313 

79 

60 

20 

39 

28 

55 

-28.5 

-79 

-32 

-46.5 

89 

238.8333 

80 

58 

0 

35 

0 

49 

2930.5 

-7115.5 

2944 

-413.667 

92 

230.8333 

81 

32 

44 

58 

62 

12 

-402.5 

-10441 

-339.5 

-3727.67 

183 

182.6667 

82 

63 

63 

63 

0 

63 

-2597 

-12641 

-2546.5 

-5928.17 

228 

118.3333 

83 

51 

61 

61 

31 

38 

-2056.5 

-2068 

-2018.5 

-2047.67 

142 

35.83333 

84 

51 

61 

61 

31 

38 

-2065 

-2085.5 

-2034.5 

-2061.67 

143 

1.833333 

85 

10 

7 

28 

63 

5 

-5684.5 

4324 

4341.5 

993.6667 

55 

-18.5 

86 

16 

29 

38 

53 

25 

1963.5 

-8068.5 

-8057 

-4720.67 

210 

-24.3333 

87 

16 

29 

38 

53 

25 

1982 

-8075 

1982.5 

-1370.17 

125 

-33.8333 

88 

0 

63 

0 

0 

63 

3759.5 

3728 

3787 

3758.167 

27 

-46.5 

89 

4 

63 

3 

17 

60 

2692.5 

-7343.5 

2714.5 

-645.5 

97 

-274.333 

90 

4 

63 

3 

17 

60 

2694 

-7357.5 

2697 

-655.5 

98 

-334.833 
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91 

63 

0 

63 

63 

0 

527 

-9495.5 

555 

-2804.5 

170 

-413.667 

92 

24 

19 

47 

53 

17 

1986 

-8043.5 

2014.5 

-1347.67 

122 

-444 

93 

28 

26 

40 

42 

25 

1952.5 

-8080.5 

1973 

-1385 

127 

-480.333 

94 

44 

52 

8 

0 

55 

2059 

-7980 

2072.5 

-1282.83 

118 

-578.5 

95 

38 

42 

20 

16 

43 

2087 

-7965 

-7950 

-4609.33 

208 

-644.667 

96 

0 

0 

63 

63 

0 

3730.5 

-6328 

3727 

376.5 

75 

-645.5 

97 

10 

7 

59 

63 

5 

2793 

-7237 

-7217.5 

-3887.17 

187 

-655.5 

98 

10 

7 

59 

63 

5 

2771.5 

-7227 

-7210 

-3888.5 

188 

-719 

99 

45 

44 

51 

23 

29 

397.5 

-9638 

415.5 

-2941.67 

174 

-736.833 

100 

45 

44 

51 

23 

29 

397 

-9635.5 

414.5 

-2941.33 

173 

-791.333 

101 

61 

62 

25 

2 

51 

-6.5 

-18.5 

30.5 

1.833333 

85 

-796.167 

102 

34 

28 

38 

56 

7 

1884 

-8164.5 

1879 

-1467.17 

131 

-867.833 

103 

47 

43 

20 

48 

4 

1884.5 

-8141.5 

1918 

-1446.33 

130 

-870.333 

104 

63 

63 

0 

0 

0 

3661.5 

-6345.5 

3708.5 

341.5 

78 

-917.333 

105 

63 

63 

0 

0 

0 

3568.5 

-6355.5 

-6347 

-3044.67 

176 

-1011.67 

106 

0 

0 

63 

63 

63 

441 

-9507 

-9494.5 

-6186.83 

230 

-1012.83 

107 

0 

0 

63 

63 

63 

647 

653.5 

653.5 

651.3333 

68 

-1035.83 

108 

10 

7 

59 

63 

37 

1252 

1234.5 

1292.5 

1259.667 

50 

-1079.5 

109 

49 

46 

60 

63 

45 

-3100 

-3134.5 

-3084.5 

-3106.33 

177 

-1107.33 

110 

12 

4 

53 

63 

0 

3474.5 

3468.5 

3510 

3484.333 

29 

-1111.5 

111 

16 

8 

54 

63 

5 

2679 

-7329.5 

-7322 

-3990.83 

193 

-1214.17 

112 

23 

59 

49 

49 

16 

169 

-9830 

229 

-3144 

178 

-1214.67 

113 

53 

59 

63 

63 

10 

-2402.5 

-12437.5 

-2384.5 

-5741.5 

226 

-1221.33 

114 

0 

62 

0 

22 

59 

2845 

-7191.5 

2905.5 

-480.333 

94 

-1263.33 

115 

0 

62 

0 

22 

59 

2916 

-7149.5 

2901.5 

-444 

93 

-1280 

116 

36 

50 

50 

51 

19 

-295 

-10338 

-245.5 

-3626.17 

182 

-1282.17 

117 

0 

0 

0 

0 

63 

6861 

-3185.5 

6914.5 

3530 

28 

-1282.83 

118 

0 

0 

0 

0 

63 

6920.5 

-3122.5 

-3105.5 

230.8333 

81 

-1285.33 

119 

63 

63 

63 

63 

0 

-2549 

-12596 

-2540.5 

-5895.17 

227 

-1308.33 

120 

63 

63 

63 

63 

0 

-2623 

-12644 

-2596.5 

-5954.5 

229 

-1337.83 

121 

51 

61 

61 

62 

7 

-2110.5 

-12137.5 

-2045 

-5431 

221 

-1347.67 

122 

28 

59 

60 

63 

8 

-885 

-884 

-842 

-870.333 

104 

-1355.33 

123 

28 

59 

60 

63 

8 

-908 

-10942 

-849 

-4233 

200 

-1360.33 

124 

49 

59 

39 

60 

13 

-977 

-11019.5 

-974.5 

-4323.67 

201 

-1370.17 

125 

49 

59 

39 

60 

13 

-998 

-11040.5 

-949 

-4329.17 

202 

-1379 

126 

27 

59 

63 

62 

13 

-1192 

-11240.5 

-1147.5 

-4526.67 

207 

-1385 

127 

40 

59 

59 

62 

35 

-2797.5 

-12795.5 

-12798 

-9463.67 

244 

-1420.33 

128 

40 

59 

59 

62 

35 

-2687 

-2738 

-2644.5 

-2689.83 

167 

-1428.33 

129 

36 

59 

59 

60 

0 

-726.5 

-10744 

-696.5 

-4055.67 

195 

-1446.33 

130 

0 

63 

0 

35 

61 

2119 

2078 

2131.5 

2109.5 

36 

-1467.17 

131 

43 

31 

42 

31 

27 

1382.5 

1347 

1400 

1376.5 

46 

-1500.83 

132 

30 

61 

20 

22 

56 

552.5 

-9484.5 

568 

-2788 

169 

-1551.33 

133 

30 

61 

20 

22 

56 

617.5 

573 

621 

603.8333 

69 

-1705.33 

134 

63 

59 

63 

0 

38 

-1070 

-1110.5 

-1058 

-1079.5 

109 

-1769 

135 

24 

62 

4 

37 

55 

915 

-9124 

924.5 

-2428.17 

163 

-1820 

136 

24 

62 

4 

37 

55 

978.5 

922.5 

975.5 

958.8333 

56 

-1864.67 

137 

0 

62 

6 

31 

57 

2279 

2244.5 

2298 

2273.833 

35 

-1883.67 

138 

26 

62 

24 

44 

58 

-746.5 

-10742.5 

-699.5 

-4062.83 

196 

-1926 
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139 

45 

63 

44 

53 

61 

-3217 

-13300 

-3180 

-6565.67 

232 

-1950.5 

140 

4 

31 

3 

17 

28 

5934.5 

5918 

5964 

5938.833 

8 

-2024.83 

141 

41 

41 

38 

22 

34 

1211.5 

-8813 

-8797.5 

-5466.33 

222 

-2047.67 

142 

41 

41 

38 

22 

34 

1208 

-8831 

1217 

-2135.33 

152 

-2061.67 

143 

61 

62 

56 

2 

51 

-1580 

-11625 

-1582.5 

-4929.17 

216 

-2062.67 

144 

16 

16 

16 

48 

16 

4434 

-5614.5 

4434.5 

1084.667 

54 

-2070.17 

145 

16 

16 

16 

48 

16 

4384 

-5616.5 

-5602.5 

-2278.33 

156 

-2077.17 

146 

33 

35 

32 

34 

25 

2051 

-7984 

2077 

-1285.33 

119 

-2084.83 

147 

62 

61 

3 

43 

0 

1586 

-8475.5 

1582.5 

-1769 

135 

-2101.83 

148 

47 

40 

46 

47 

47 

-1397 

-11397 

-11390 

-8061.33 

237 

-2108.83 

149 

17 

24 

18 

17 

17 

5423 

5371 

5460.5 

5418.167 

14 

-2123 

150 

17 

24 

18 

17 

17 

5323 

-4677.5 

-4659 

-1337.83 

121 

-2130 

151 

60 

59 

18 

28 

13 

1105 

-8937 

1111 

-2240.33 

155 

-2135.33 

152 

63 

0 

0 

63 

0 

3664.5 

-6341 

-6334.5 

-3003.67 

175 

-2205.67 

153 

62 

29 

2 

52 

1 

2675.5 

-7333 

2723.5 

-644.667 

96 

-2225.5 

154 

58 

60 

53 

0 

41 

-640.5 

-10647.5 

-591.5 

-3959.83 

192 

-2240.33 

155 

58 

60 

51 

2 

39 

-480 

-10524 

-475 

-3826.33 

186 

-2278.33 

156 

60 

61 

54 

23 

46 

-2182 

-12224.5 

-2175.5 

-5527.33 

225 

-2312.5 

157 

61 

62 

58 

43 

55 

-3905 

-3928 

-3882 

-3905 

190 

-2325 

158 

29 

30 

25 

2 

19 

4831 

4823 

4864 

4839.333 

18 

-2326.5 

159 

29 

30 

25 

2 

19 

4706.5 

-5288.5 

-5269.5 

-1950.5 

140 

-2327.83 

160 

62 

58 

15 

46 

19 

0.5 

-10027 

58.5 

-3322.67 

180 

-2376.67 

161 

11 

10 

11 

8 

0 

7950 

-2054 

-2040.5 

1285.167 

48 

-2380.17 

162 

11 

10 

15 

0 

0 

-1942 

-1834.5 

-1817.5 

-1864.67 

137 

-2428.17 

163 

2 

4 

4 

34 

1 

-2410 

-2295 

-2270 

-2325 

158 

-2558.5 

164 

2 

4 

4 

34 

1 

-2319.5 

7722 

7737 

4379.833 

21 

-2665.5 

165 

11 

9 

9 

16 

0 

-2389 

-2285 

-2263.5 

-2312.5 

157 

-2672 

166 

12 

13 

16 

0 

0 

-2166.5 

-2080 

-2059 

-2101.83 

148 

-2689.83 

167 

0 

0 

0 

11 

62 

-3813.5 

-3694.5 

-3676.5 

-3728.17 

184 

-2752.5 

168 

8 

4 

7 

5 

13 

8259 

-1817.5 

-1801 

1546.833 

43 

-2788 

169 

8 

4 

7 

5 

13 

-1932 

-1868.5 

-1850.5 

-1883.67 

138 

-2804.5 

170 

13 

11 

16 

1 

0 

-2176 

-2084 

-2066.5 

-2108.83 

149 

-2806 

171 

13 

11 

16 

1 

0 

-2210 

-2088.5 

-2070.5 

-2123 

150 

-2874 

172 

10 

17 

12 

2 

13 

-2809.5 

-2732 

-2716 

-2752.5 

168 

-2941.33 

173 

12 

3 

14 

0 

0 

8463.5 

-1482 

-1461 

1840.167 

40 

-2941.67 

174 

3 

38 

2 

18 

41 

-5225 

-5134 

-5112.5 

-5157.17 

219 

-3003.67 

175 

9 

7 

10 

1 

0 

8694 

-1350.5 

-1341 

2000.833 

38 

-3044.67 

176 

13 

13 

16 

1 

0 

-2268.5 

-2188 

-2160.5 

-2205.67 

153 

-3106.33 

177 

11 

16 

15 

3 

0 

-2415.5 

-2291 

-2273 

-2326.5 

159 

-3144 

178 

11 

16 

15 

3 

0 

-2413 

-2297 

-2273.5 

-2327.83 

160 

-3186.83 

179 

11 

4 

11 

0 

0 

-1418 

-1332.5 

-1315.5 

-1355.33 

123 

-3322.67 

180 

8 

9 

8 

12 

15 

7367 

-2630.5 

-2612 

708.1667 

63 

-3412.83 

181 

8 

9 

8 

12 

15 

7341.5 

-2618.5 

-2602.5 

706.8333 

64 

-3626.17 

182 

14 

11 

18 

0 

0 

-2317 

-2186 

-2173.5 

-2225.5 

154 

-3727.67 

183 

14 

6 

13 

5 

15 

7478 

-2606 

-2601.5 

756.8333 

58 

-3728.17 

184 

8 

14 

13 

0 

0 

8179 

-1775.5 

-1754 

1549.833 

42 

-3792.33 

185 

8 

14 

13 

0 

0 

-1904 

-1789.5 

-1766.5 

-1820 

136 

-3826.33 

186 

8 

14 

9 

4 

6 

8019.5 

-2032.5 

8017 

4668 

19 

-3887.17 
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187 

6 

17 

6 

8 

13 

-2633 

-2530.5 

-2512 

-2558.5 

164 

-3888.5 

188 

6 

17 

6 

8 

13 

7431 

-2524 

-2507 

800 

57 

-3903.83 

189 

1 

3 

4 

21 

19 

7634.5 

-2396.5 

7658 

4298.667 

22 

-3905 

190 

1 

1 

2 

15 

37 

-2826 

-2809.5 

-2782.5 

-2806 

171 

-3958.5 

191 

0 

0 

0 

5 

59 

-3202 

-3192 

-3166.5 

-3186.83 

179 

-3959.83 

192 

33 

34 

35 

45 

1 

2668.5 

-7374 

-7352 

-4019.17 

194 

-3990.83 

193 

33 

34 

35 

45 

1 

2554 

-7447 

-7433 

-4108.67 

197 

-4019.17 

194 

1 

1 

2 

9 

0 

-702.5 

9322 

9343.5 

5987.667 

7 

-4055.67 

195 

33 

34 

35 

45 

32 

-9225.5 

-9228 

-9170.5 

-9208 

242 

-4062.83 

196 

3 

6 

5 

26 

9 

7589.5 

7564.5 

7626 

7593.333 

3 

-4108.67 

197 

1 

2 

7 

28 

0 

-1963 

-1916 

-1899 

-1926 

139 

-4184.83 

198 

5 

33 

2 

16 

36 

-4655 

5371 

5387 

2034.333 

37 

-4221 

199 

1 

3 

4 

19 

4 

8485.5 

-1557.5 

-1542.5 

1795.167 

41 

-4233 

200 

1 

3 

4 

19 

4 

-1596 

8426 

8442.5 

5090.833 

15 

-4323.67 

201 

0 

0 

0 

0 

15 

-832.5 

-781.5 

-760 

-791.333 

101 

-4329.17 

202 

3 

10 

11 

22 

7 

-2658 

7338 

7359.5 

4013.167 

23 

-4416.83 

203 

18 

30 

23 

12 

23 

-5336 

-5571.5 

-5524 

-5477.17 

223 

-4454.83 

204 

2 

3 

5 

29 

28 

6637.5 

-3378.5 

-3360.5 

-33.8333 

88 

-4490.33 

205 

2 

5 

7 

25 

0 

8068.5 

-1951 

-1924.5 

1397.667 

44 

-4514 

206 

3 

0 

0 

34 

63 

-5080.5 

4971.5 

-5007 

-1705.33 

134 

-4526.67 

207 

12 

3 

7 

23 

15 

7127 

-2960.5 

-2946.5 

406.6667 

74 

-4609.33 

208 

12 

3 

7 

23 

15 

6947 

-3015 

-2993 

313 

79 

-4696.83 

209 

32 

1 

14 

13 

63 

-6215.5 

3823 

3838.5 

482 

71 

-4720.67 

210 

32 

1 

14 

13 

63 

3912 

-6135.5 

3911.5 

562.6667 

70 

-4760.33 

211 

2 

4 

7 

29 

0 

-2160 

-2125.5 

-2104.5 

-2130 

151 

-4780 

212 

3 

21 

1 

9 

25 

-3027.5 

7021 

-2962 

343.8333 

77 

-4798.83 

213 

3 

21 

1 

9 

25 

7010.5 

-2962 

-2940 

369.5 

76 

-4899.5 

214 

0 

0 

0 

10 

58 

6664.5 

-3380 

-3357.5 

-24.3333 

87 

-4912.5 

215 

0 

0 

0 

10 

58 

6665.5 

-3371.5 

-3349.5 

-18.5 

86 

-4929.17 

216 

0 

8 

1 

12 

52 

6414 

-3629.5 

-3607.5 

-274.333 

90 

-4962.17 

217 

0 

0 

0 

6 

58 

6892 

-3182 

-3162 

182.6667 

82 

-5052.67 

218 

4 

39 

4 

20 

25 

5468 

-4574.5 

-4557.5 

-1221.33 

114 

-5157.17 

219 

4 

39 

4 

20 

25 

5375 

-4614 

-4601 

-1280 

116 

-5365.67 

220 

0 

0 

0 

9 

50 

7113.5 

-2930 

-2908 

425.1667 

73 

-5431 

221 

0 

0 

0 

7 

7 

9366 

-671 

-649 

2682 

32 

-5466.33 

222 

0 

0 

0 

7 

7 

-717 

-969.5 

-917 

-867.833 

103 

-5477.17 

223 

6 

8 

6 

12 

42 

6364.5 

-3694 

-3675 

-334.833 

91 

-5495.67 

224 

40 

44 

41 

40 

9 

1311 

-8713.5 

-8694.5 

-5365.67 

220 

-5527.33 

225 

1 

0 

0 

17 

58 

6163 

-3847 

6213 

2843 

31 

-5741.5 

226 

0 

0 

0 

1 

52 

7453.5 

-2620.5 

-2604 

743 

60 

-5895.17 

227 

4 

0 

1 

40 

63 

4660 

-5375.5 

-5359 

-2024.83 

141 

-5928.17 

228 

4 

0 

1 

40 

63 

4725 

-5360 

4694.5 

1353.167 

47 

-5954.5 

229 

0 

0 

0 

8 

55 

6950 

-3129 

-3104.5 

238.8333 

80 

-6186.83 

230 

29 

2 

18 

18 

54 

4017 

-6025 

-6008 

-2672 

166 

-6398.5 

231 

29 

2 

18 

18 

54 

4005.5 

-6040.5 

4014.5 

659.8333 

66 

-6565.67 

232 

2 

14 

1 

10 

31 

7130 

-2890 

-2869 

457 

72 

-6894 

233 

33 

42 

33 

37 

10 

2235.5 

-7756 

-7730 

-4416.83 

203 

-7188.5 

234 

33 

42 

33 

37 

10 

2213.5 

-7796 

-7782 

-4454.83 

204 

-7366.5 
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235 

8 

63 

6 

35 

57 

-8515 

-8475.5 

-8454.5 

-8481.67 

239 

-7974.5 

236 

6 

47 

5 

26 

42 

3780 

3734 

3785.5 

3766.5 

26 

-8061.33 

237 

0 

0 

0 

2 

4 

9720.5 

-300.5 

9752.5 

6390.833 

6 

-8265.83 

238 

0 

0 

0 

9 

16 

-1514.5 

-1522.5 

-1465.5 

-1500.83 

132 

-8481.67 

239 

1 

4 

3 

5 

7 

-1008 

8989 

9010.5 

5663.833 

13 

-8560.83 

240 

18 

28 

21 

22 

24 

-6534.5 

4327 

4349 

713.8333 

62 

-8720.17 

241 

18 

28 

21 

22 

24 

4187.5 

-5670 

-5658 

-2380.17 

162 

-9208 

242 

2 

0 

1 

21 

42 

-3308.5 

-3297.5 

6713.5 

35.83333 

84 

-9293.33 

243 

63 

0 

63 

63 

63 

-12878.5 

-12582.5 

-12565.5 

-12675.5 

248 

-9463.67 

244 

33 

0 

32 

48 

63 

-9714 

1264.5 

1319.5 

-2376.67 

161 

-10384.7 

245 

7 

1 

4 

6 

18 

-1996 

8244.5 

8298.5 

4849 

17 

-10639.2 

246 

15 

1 

9 

11 

36 

-4514 

-3610 

-3587.5 

-3903.83 

189 

-12099.5 

247 

43 

33 

38 

40 

59 

-10637.5 

-10649.5 

-10630.5 

-10639.2 

246 

-12675.5 

248 

0 

7 

1 

5 

3 

-780.5 

9266.5 

9321.5 

5935.833 

9 

-12675.5 

249 

0 

7 

1 

5 

3 

9283.5 

-1072.5 

9192 

5801 

10 

-16220.3 
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Appendix 4: Experiment with 689 runs 


Aircraft Number by Type Fitness Value Total 


Trials 

A1 

A2 

A3 

A4 

A5 

Scenariol 

Scenario2 

Scenario3 

Fitness 

rank 

sorted 

1 

0 

0 

0 

0 

0 

10014 

- 19.5 

10030 

6674.833 

12 

9342.167 

2 

63 

63 

63 

63 

63 

- 16017.5 

-16027 

-16000 

- 16014.8 

689 

8561 

3 

3 

0 

1 

25 

55 

-4450 

5866 

5919 

2445 

112 

8262 

4 

61 

58 

3 

41 

1 

- 8475.5 

- 8171.5 

- 8145.5 

- 8264.17 

654 

8094.167 

5 

8 

62 

5 

34 

56 

- 9152.5 

-8301 

- 8293.5 

- 8582.33 

660 

8055.333 

6 

59 

6 

37 

2 

52 

- 7972.5 

2234.5 

2292 

- 1148.67 

305 

7209.833 

7 

38 

59 

59 

61 

13 

- 1632.5 

-11505 

- 1459.5 

- 4865.67 

578 

7149.167 

8 

58 

60 

49 

3 

38 

-395 

-10441 

- 349.5 

- 3728.5 

527 

6939.667 

9 

20 

13 

55 

63 

10 

2000.5 

-8070 

1979 

- 1363.5 

327 

6911 

10 

0 

0 

0 

7 

12 

8915.5 

- 976.5 

9078 

5672.333 

32 

6875.833 

11 

3 

0 

1 

32 

63 

-4963 

5034.5 

5057 

1709.5 

148 

6803.833 

12 

14 

3 

10 

1 

22 

7507 

-2460 

-2451 

865.3333 

185 

6674.833 

13 

63 

9 

47 

3 

63 

705.5 

-9266 

-9246 

- 5935.5 

606 

6372.833 

14 

2 

4 

6 

27 

1 

8044.5 

8034 

8087.5 

8055.333 

5 

6330.667 

15 

22 

17 

61 

63 

11 

-8761 

1273 

1288.5 

- 2066.5 

395 

6223.333 

16 

19 

20 

16 

1 

13 

6551 

- 3481.5 

6577.5 

3215.667 

81 

6209.167 

17 

39 

40 

33 

2 

25 

2926.5 

- 6984.5 

-6966 

- 3674.67 

524 

6202.667 

18 

61 

62 

56 

33 

51 

-3169 

-13185 

-13164 

- 9839.33 

670 

6177.5 

19 

13 

20 

20 

20 

4 

6131 

6171 

6220 

6174 

20 

6175.167 

20 

25 

39 

39 

41 

9 

- 8552.5 

-7686 

- 7668.5 

-7969 

649 

6174 

21 

51 

61 

61 

62 

38 

-3779 

- 13694.5 

- 13680.5 

- 10384.7 

675 

6032.333 

22 

11 

10 

13 

1 

0 

-1929 

8312.5 

8358 

4913.833 

42 

5984.167 

23 

43 

42 

45 

33 

32 

358 

- 9788.5 

- 9766.5 

-6399 

620 

5873.333 

24 

27 

1 

1 

13 

0 

7915 

- 2122.5 

7965.5 

4586 

52 

5833 

25 

63 

59 

4 

54 

1 

-9208 

- 9087.5 

-9063 

- 9119.5 

664 

5824.333 

26 

2 

17 
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0 

4 

4 

36 

0 

-2319 

- 2225.5 

- 2206.5 

- 2250.33 

419 

- 8465.33 

659 

38 

4 

13 

0 

20 

-3809 

6223 

6239 

2884.333 

94 

- 8516.67 

660 

2 

5 

12 

27 

6 

7340 

- 2628.5 

- 2613.5 

699.3333 

192 

- 8582.33 

661 

2 

7 

18 

26 

12 

- 3328.5 

- 3276.5 

-3253 

-3286 

499 

-8773 

662 

2 

63 

63 

0 

63 

394.5 

- 9562.5 

-9546 

-6238 

615 

-8855 

663 

2 

36 

44 

13 

40 

3293.5 

-6749 

3296.5 

-53 

225 

- 8928.5 

664 

2 

0 

0 

63 

0 

6775.5 

- 3260.5 

- 3241.5 

91.16667 

215 

- 9119.5 

665 

2 

2 

3 

45 

1 

- 2699.5 

7322.5 

7343.5 

3988.833 

68 

- 9265.17 

666 

5 

10 

6 

26 

28 

- 3804.5 

6221 

6241 

2885.833 

93 

- 9308.67 

667 

0 

0 

6 

28 

0 

8342.5 

- 1707.5 

- 1680.5 

1651.5 

150 

- 9535.5 

668 

27 

63 

9 

16 

63 

- 8949.5 

1069.5 

1091.5 

- 2262.83 

420 

-9541 

669 

20 

26 

8 

29 

4 

5696 

5680.5 

5727.5 

5701.333 

28 

- 9725.83 

670 

0 

0 

4 

25 

0 

8498 

-1487 

-1469 

1847.333 

140 

- 9839.33 

671 

63 

63 

17 

56 

10 

-10494 

-477 

- 10673.5 

- 7214.83 

637 

- 9854.5 

672 

3 

9 

13 

24 

2 

7441 

- 2585.5 

7500.5 

4118.667 

67 

- 9873.67 

673 

1 

0 

0 

30 

0 

-1658 

- 1586.5 

- 1558.5 

-1601 

353 

- 9982.67 

674 

18 

34 

36 

14 

5 

-5383 

- 5625.5 

-5573 

- 5527.17 

594 

- 10028.3 

675 

0 

0 

44 

3 

6 

7279.5 

- 2692.5 

- 2675.5 

637.1667 

195 

- 10384.7 

676 

9 

1 

46 

3 

5 

-3334 

-3234 

-3214 

- 3260.67 

497 

- 10593.3 

677 

17 

3 

28 

6 

10 

6688.5 

- 3239.5 

-3219 

76.66667 

216 

- 10639.2 

678 

45 

34 

6 

36 

39 

1901.5 

-8039 

-8013 

- 4716.83 

569 

-10661 

679 

0 

0 

52 

5 

1 

7145.5 

7120.5 

7181.5 

7149.167 

7 

- 10760.8 

680 

8 

21 

45 

6 

21 

- 5205.5 

-5104 

- 5066.5 

- 5125.33 

585 

-10816 

681 

16 

42 

27 

11 

42 

3061 

- 6922.5 

- 6910.5 

- 3590.67 

518 

- 10872.7 

682 

16 

21 

47 

18 

2 

4827.5 

- 5208.5 

4841.5 

1486.833 

156 

- 11075.2 

683 

32 

42 

31 

36 

5 

2618.5 

-7346 

- 7328.5 

- 4018.67 

543 

- 11082.5 

684 

56 

63 

8 

59 

35 

- 1093.5 

-11090 

- 11086.5 

- 7756.67 

648 

- 11622.3 

685 

4 

5 

51 

3 

0 

6911 

6868 

6954 

6911 

9 

-12596 

686 

9 

1 

4 

9 

42 

- 3424.5 

- 3301.5 

-3277 

- 3334.33 

505 

- 12925.3 

687 

17 

3 

7 

9 

28 

6851.5 

-3191 

-3170 

163.5 

211 

- 14213.8 

688 

63 

63 

63 

9 

0 

67.5 

- 9907.5 

-9887 

- 6575.67 

624 

- 15275.5 

689 

45 

34 

37 

9 

8 

3263.5 

- 6699.5 

- 6695.5 

- 3377.17 

508 

- 16014.8 
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Appendix 5: Regression Analysis 


The whole model includes parameters polynomial to fifth degree and full factorial. The 
variability in the data is explained 57% despite the huge number of the parameters. Each 
complete run among the others was performed separately and under the same conditions, which 
provides the error terms to be independently identically distributed, and Shapiro-Wilkinson test 
verified that they are also normal. Residual plot shows no significant pattern but an even 
distribution and constant variance. The details are shown below: 


Response: weighted objective 
Summary of Fit 

RSquare 
RSquare Adj 
Root Mean Square Error 
Mean of Response 
Observations (or Sum Wgts) 

Lack of Fit 

Source DF Sum of Squares 

Lack of Fit 147 1284339016 

Pure Error 50 256483011 

Total Error 197 1540822027 

Max RSq 
0.9289 


Parameter Estimates 


Term 

Estimate 

Std Error 

t Ratio 

Prob>|t| 

Intercept 

5952.1424 

1520.016 

3.92 

0.0001 

Aircraftl 

-750.7417 

361.5091 

-2.08 

0.0391 

Aircraft2 

90.399428 

330.1415 

0.27 

0.7845 

Aircraftl *Aircraft2 

2.9002642 

3.249142 

0.89 

0.3731 

Aircraft3 

-348.1698 

356.4375 

-0.98 

0.3299 

Aircraftl *Aircraft3 

7.3422125 

5.308086 

1.38 

0.1682 

Aircraft2*Aircraft3 

11.365968 

10.67632 

1.06 

0.2884 

Aircraftl *Aircraft2*Aircraft3 

-0.223981 

0.168747 

-1.33 

0.1859 

Aircraft4 

-684.4209 

291.7342 

-2.35 

0.0200 

Aircraftl *Aircraft4 

1.4355941 

3.089653 

0.46 

0.6427 

Aircraft2*Aircraft4 

1.9149344 

7.350865 

0.26 

0.7947 

Aircraftl *Aircraft2*Aircraft4 

-0.041533 

0.113525 

-0.37 

0.7149 

Aircraft3*Aircraft4 

4.4079979 

5.189752 

0.85 

0.3967 

Aircraftl *Aircraft3*Aircraft4 

-0.10655 

0.092628 

-1.15 

0.2514 

Aircraft2*Aircraft3*Aircraft4 

-0.187256 

0.189892 

-0.99 

0.3253 

Aircraftl *Aircraft2*Aircraft3*Aircraft4 

0.0034392 

0.002924 

1.18 

0.2409 

Aircraft5 

-106.3785 

299.3437 

-0.36 

0.7227 

Aircraftl *Aircraft5 

6.3811637 

3.051069 

2.09 

0.0378 

Aircraft2*Aircraft5 

2.5785061 

2.604744 

0.99 

0.3234 

Aircraftl *Aircraft2*Aircraft5 

-0.124258 

0.066351 

-1.87 

0.0626 

Aircraft3*Aircraft5 

1.6071497 

6.294813 

0.26 

0.7987 

Aircraftl *Aircraft3*Aircraft5 

-0.17233 

0.095284 

-1.81 

0.0720 

Aircraft2*Aircraft3*Aircraft5 

-0.095878 

0.239275 

-0.40 

0.6891 

Aircraftl *Aircraft2*Aircraft3*Aircraft5 

0.0038655 

0.003666 

1.05 

0.2929 

Aircraft4*Aircraft5 

1.8638123 

1.562528 

1.19 

0.2344 

Aircraftl *Aircraft4*Aircraft5 

-0.041985 

0.09958 

-0.42 

0.6738 

Aircraft2*Aircraft4*Aircraft5 

-0.057478 

0.123623 

-0.46 

0.6425 


0.572761 

0.462156 

2796.682 

-1400.57 


249 


Mean Square 

F Ratio 

8737000 

1.7032 

5129660 

Prob>F 

0.0157 
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Aircraft1*Aircraft2*Aircraft4*Aircraft5 


0.0007991 

0.002316 

0.34 

0.7305 


Aircraft3*Aircraft4*Aircraft5 


-0.035482 

0.103133 

-0.34 

0.7312 


Aircraft1*Aircraft3*Aircraft4*Aircraft5 


0.001326 

0.002295 

0.58 

0.5641 


Aircraft2*Aircraft3*Aircraft4*Aircraft5 


0.0019465 

0.004276 

0.46 

0.6495 


Aircraft1*Aircraft2*Aircrafl3*Aircraft4*Aircraft5 


-0.000043 

0.00007 

-0.62 

0.5359 


Aircraftl *Aircraft1 


48.61383 

36.03725 

1.35 

0.1789 


Aircraftl *Aircraft1 *Aircraft1 


-1.664394 

1.519078 

-1.10 

0.2746 


Aircraftl *Aircraft1 *Aircraft1 *Aircraft1 


0.0261299 

0.027322 

0.96 

0.3401 


Aircraftl *Aircraft1 *Aircraft1 *Aircraft1 *Aircraft1 


-0.000153 

0.000175 

-0.87 

0.3828 


Aircraft2*Aircraft2 


-7.313304 

33.80103 

-0.22 

0.8289 


Aircraft2*Aircraft2*Aircraft2 


-0.109586 

1.439094 

-0.08 

0.9394 


Aircraft2*Aircraft2*Aircraft2*Aircraft2 


0.005922 

0.026153 

0.23 

0.8211 


Aircraft2*Aircraft2*Aircraft2*Aircraft2*Aircraft2 


-0.000051 

0.00017 

-0.30 

0.7622 


Aircraft3*Aircraft3 


13.002956 

38.23385 

0.34 

0.7342 


Ai rcraft3*Ai rcraft3*A i rcraft3 


-0.968784 

1.515205 

-0.64 

0.5233 


Ai rcraft3*Ai rcraft3* Ai rcraft3*Ai rcraft 3 


0.0224916 

0.026537 

0.85 

0.3977 


Aircraft3*Aircraft3*Aircraft3*Aircraft3*Aircraft3 


-0.000164 

0.000167 

-0.98 

0.3294 


Aircraft4*Aircraft4 


70.240404 

31.28921 

2.24 

0.0259 


Aircraft4*Aircraft4*Aircraft4 


-2.864449 

1.375211 

-2.08 

0.0385 


Aircraft4*Aircraft4*Aircraft4*Aircraft4 


0.048631 

0.025701 

1.89 

0.0599 


Aircraft4*Aircraft4*Aircraft4*Aircraft4*Aircraft4 


-0.000291 

0.00017 

-1.71 

0.0894 


Aircraft5*Aircraft5 


17.33458 

33.67648 

0.51 

0.6073 


Aircraft5*Aircraft5*Aircraft5 


-1.170897 

1.43315 

-0.82 

0.4149 


Aircraft5*Aircraft5*Aircraft5*Aircraft5 


0.0262925 

0.025463 

1.03 

0.3031 


Aircraft5*Aircraft5*Aircraft5*Aircraft5*Aircraft5 

Effect Test 

-0.000191 

0.00016 

-1.20 

0.2331 


Source 

Nparm DF 

Sum of Squares 


F Ratio 

Prob>F 

Aircraftl 


1 1 

33730973 


4.3126 

0.0391 

Aircraft2 


1 1 

586432 


0.0750 

0.7845 

Aircraftl *Aircraft2 


1 1 

6231949 


0.7968 

0.3731 

Aircraft3 


1 1 

7462795 


0.9541 

0.3299 

Aircraftl *Aircraft3 


1 1 

14964568 


1.9133 

0.1682 

Aircraft2*Aircraft3 


1 1 

8864541 


1.1334 

0.2884 

Aircraftl *Aircraft2*Aircraft3 


1 1 

13779692 


1.7618 

0.1859 

Aircraft4 


1 1 

43048494 


5.5039 

0.0200 

Aircraftl *Aircraft4 


1 1 

1688613 


0.2159 

0.6427 

Aircraft2*Aircraft4 


1 1 

530783 


0.0679 

0.7947 

Aircraftl *Aircraft2*Aircraft4 


1 1 

1046876 


0.1338 

0.7149 

Aircraft3*Aircraft4 


1 1 

5642555 


0.7214 

0.3967 

Aircraftl *Aircraft3*Aircraft4 


1 1 

10349239 


1.3232 

0.2514 

Aircraft2*Aircraft3*Aircraft4 


1 1 

7605765 


0.9724 

0.3253 

Aircraftl *Aircraft2*Aircraft3*Aircraft4 


1 1 

10821652 


1.3836 

0.2409 

Aircraft5 


1 1 

987765 


0.1263 

0.7227 

Aircraftl *Aircraft5 


1 1 

34212266 


4.3742 

0.0378 

Aircraft2*Aircraft5 


1 1 

7664653 


0.9800 

0.3234 

Aircraftl *Aircraft2*Aircraft5 


1 1 

27431227 


3.5072 

0.0626 

Aircraft3*Aircraft5 


1 1 

509839 


0.0652 

0.7987 

Aircraftl *Aircraft3*Aircraft5 


1 1 

25584250 


3.2710 

0.0720 

Aircraft2*Aircraft3*Aircraft5 


1 1 

1255831 


0.1606 

0.6891 

Aircraft1*Aircraft2*Aircraft3*Aircraft5 


1 1 

8697478 


1.1120 

0.2929 

Aircraft4*Aircraft5 


1 1 

11128464 


1.4228 

0.2344 

Aircraftl *Aircraft4*Aircraft5 


1 1 

1390326 


0.1778 

0.6738 

Aircraft2*Aircraft4*Aircraft5 


1 1 

1690809 


0.2162 

0.6425 

Aircraftl *Aircraft2*Aircraft4*Aircraft5 


1 1 

930883 


0.1190 

0.7305 

Aircraft3*Aircraft4* Aircrafts 


1 1 

925766 


0.1184 

0.7312 

Aircraftl *Aircraft3*Aircraft4*Aircraft5 


1 1 

2611136 


0.3338 

0.5641 

Aircraft2*Aircraft3*Aircraft4*Aircraft5 


1 1 

1620813 


0.2072 

0.6495 

Aircraft1*Aircraft2*Aircraft3*Aircraft4*Aircraft5 


1 1 

3007214 


0.3845 

0.5359 

Aircraftl *Aircraft1 


1 1 

14233202 


1.8198 

0.1789 

Aircraftl *Aircraft1 *Aircraft1 


1 1 

9389411 


1.2005 

0.2746 

Aircraftl *Aircraft1 *Aircraft1 *Aircraft1 


1 1 

7153815 


0.9146 

0.3401 
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Aircraftl *Aircraft1 *Aircraft1 *Aircraft1 *Aircraft1 

Aircraft2*Aircraft2 

Aircraft2*Aircraft2*Aircraft2 

Aircraft2*Aircraft2*Aircraft2*Aircraft2 

Aircraft2*Aircraft2*Aircraft2*Aircraft2*Aircraft2 

Aircraft3*Aircraft3 

Aircraft3*Aircraft3*Aircraft3 

Aircraft3*Aircraft3*Aircraft3*Aircraft3 

Aircraft3*Aircraft3*Aircraft3*Aircraft3*Aircraft3 

Aircraft4*Aircraft4 

Ai rcraft4*Ai rcraft4*Ai rcraft4 

Aircraft4*Aircraft4*Aircraft4*Aircraft4 

Aircraft4*Aircraft4*Aircraft4*Aircraft4*Aircraft4 

Aircraft5*Aircraft5 

Ai rcraft5*Ai rcraft 5*Ai rcraftS 

Aircraft5*Aircraft5*Aircraft5*Aircraft5 

Aircraft5*Aircraft5*Aircraft5*Aircraft5*Aircraft5 


1 

1 

5984250 

0.7651 

1 

1 

366145 

0.0468 

1 

1 

45354 

0.0058 

1 

1 

401031 

0.0513 

1 

1 

718329 

0.0918 

1 

1 

904637 

0.1157 

1 

1 

3197399 

0.4088 

1 

1 

5618498 

0.7183 

1 

1 

7476544 

0.9559 

1 

1 

39415871 

5.0395 

1 

1 

33933558 

4.3385 

1 

1 

28002853 

3.5803 

1 

1 

22793760 

2.9143 

1 

1 

2072333 

0.2650 

1 

1 

5220840 

0.6675 

1 

1 

8339435 

1.0662 

1 

1 

11188150 

1.4304 



0.3828 

0.8289 

0.9394 

0.8211 

0.7622 

0.7342 

0.5233 

0.3977 

0.3294 

0.0259 

0.0385 

0.0599 

0.0894 

0.6073 

0.4149 

0.3031 

0.2331 
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maximum 


quartile 

median 

quartile 


minimum 


Quantiles 

100 . 0 % 

99.5% 

97.5% 

90.0% 

75.0% 

50.0% 

25.0% 

10 . 0 % 

2.5% 

0.5% 

0.0% 


8390.9 

7856.2 

5455.7 

3339.7 
1610.6 

-63.0 

-1534.6 

-2973.6 

-4474.0 

-6208.4 

-6281.9 


- 0.000 

2492.587 

157.961 

311.121 

-311.121 

249.000 

249.000 


Moments 

Mean 

Std Dev 

Std Error Mean 

Upper 95% Mean 

Lower 95% Mean 

N 

Sum Weights 


Test for Normality 
Shapiro-Wilk WTest 

W Prob<W 
0.982146 0.4106 
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To reduce the number of parameters, we run stepwise regression. We tested the 
assumptions of the resulting model. The error terms were independently identically 
distributed, and their normality is verified by Shapiro-Wilkinson test. The model residuals 
show no significant pattern and the constancy of variance is verified by Breush-Pegan test. 


Analysis of Variance 


Source 

DF 

Sum of Squares 

Mean Square 

F Ratio 

Model 

8 

1.96969e15 

2.462e14 

2.6492 

Error 

240 

2.2305e+16 

9.294e13 

Prob>F 

C Total 

248 

2.42747e16 


0.0084 


The above ANOVA table obtained by regressing the squared residuals over the reduced 
model is used in testing the variance. The chi-square statistic obtained is 16,4. This is less than 
20.09 at .01 significance level showing constant variance. In addition, given that the whole 
model could explain the variability in the data 57%, the reduced model performs well by 46%. 
Here is the model: 

y = 2136.4525 + 10.198999 Aircraftl- 33.97995 Aircraft- 91.29319 Aircraft3+ 
7.0619104 Aircraft4-1.38817 Aircraftl *Aircraft4-15.1685 Aircraft5- 
0.816226Aircraft3*Aircraft5+ 0.9511885Aircraft3*Aircraft3 + error 

By examining the stationary points of this function, we find that maximum values are located 
around xi=5 and X 4 = 7 . The approximation function has other stationary points in areas where 
X 3 and X 5 take negative values. 

The details of the regression are given below: 


Response: weighted objective 
Summary of Fit 


RSquare 0.464923 

RSquare Adj 0.447087 

Root Mean Square Error 2835.591 

Mean of Response -1400.57 

Observations (or Sum Wgts) 249 


Lack of Fit 


Source 

DF 

Sum of Squares 

Mean Square F Ratio 

Lack of Fit 

190 

1673254879 

88066051.7168 

Pure Error 

50 

256483011 

5129660 Prob>F 

Total Error 

240 

1929737890 

0.0130 


Max RSq 
0.9289 


Parameter Estimates 


Term 

Estimate 

Std Error 

t Ratio 

Prob>|t| 

Intercept 

2136.4525 

542.6446 

3.94 

0.0001 

Aircraftl 

10.198999 

16.99174 

0.60 

0.5489 

Aircraft2 

-33.97995 

9.142528 

-3.72 

0.0003 

Aircraft3 

-91.29319 

39.37482 

-2.32 

0.0213 

Aircraft4 

7.0619104 

17.89208 

0.39 

0.6934 

Aircraftl *Aircraft4 

-1.38817 

0.423166 

-3.28 

0.0012 

Aircraft5 

-15.1685 

11.42189 

-1.33 

0.1854 
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Aircraft3*Aircraft5 


-0.816226 0.398438 

-2.05 

0.0416 

Aircraft3*Aircraft3 


0.9511885 0.562623 

1.69 

0.0922 



Effect Test 



Source 

Nparm 

DF 

Sum of Squares 

F Ratio 

Prob>F 

Aircraftl 

1 

1 

2896852 

0.3603 

0.5489 

Aircraft2 

1 

1 

111070825 

13.8138 

0.0003 

Aircraft3 

1 

1 

43224180 

5.3758 

0.0213 

Aircraft4 

1 

1 

1252593 

0.1558 

0.6934 

Aircraftl *Aircraft4 

1 

1 

86526933 

10.7613 

0.0012 

Aircrafts 

1 

1 

14180661 

1.7636 

0.1854 

Aircraft3*Aircraft5 

1 

1 

33743262 

4.1966 

0.0416 

Aircraft3*Aircraft3 

1 

1 

22981843 

2.8582 

0.0922 



Analysis of Variance 


Source 

DF 

Sum of Squares 

Mean Square 

F Ratio 

Model 

8 

1676727139 

2.0959e8 

26.0667 

Error 

240 

1929737890 

8040575 

Prob>F 

C Total 

248 

3606465029 


<.0001 
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Residual weighted object 



-j-|-1-1-1-j 

- 3 - 2-10123 


Normal Quantile 


Quantiles 

maximum 100.0% 

7439.1 

99.5% 

7192.4 

97.5% 

5535.8 

90.0% 

4013.4 

quartile 75.0% 

1472.8 

median 50.0% 

156.1 

quartile 25.0% 

-2022.1 

10.0% 

-3549.6 

2.5% 

-5479.4 

0.5% 

-6561.0 

minimum 0.0% 

-6760.2 

Moments 

Mean 

- 0.000 

Std Dev 

2789.480 

Std Error Mean 

176.776 

Upper 95% Mean 

348.178 

Lower 95% Mean 

-348.178 

N 

249.000 

Sum Weights 

249.000 


Test for Normality 
Shapiro-Wilk W Test 

W Prob<W 
0.975069 0.0559 
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Parameter estimates below describe the model obtained by regressing the local optima 
of multi-scenario objective function values over explanatory variables, i.e., aircraft types. 
Examining the stationary points, we have found that this regression function has a maximum 
where X 3 is close to 18. 


Response: weighted objective 
Summary of Fit 



RSquare 



0.95112 



RSquare Adj 


0.918534 



Root Mean Square Error 

1045.427 



Mean of Response 

2966.439 



Observations (or Sum Wgts) 


11 



Parameter Estimates 



Term 


Estimate 

Std Error 

t Ratio 

Prob>|t| 

Intercept 


7389.4478 

808.5816 

9.14 

<.0001 

Aircraftl 


-272.3735 

33.55304 

-8.12 

0.0002 

Alrcraft2 


53.682613 

42.00488 

1.28 

0.2485 

Aircraft3 


211.52482 

43.80634 

4.83 

0.0029 

Aircraft2*Aircraft3 

-3.19308 

0.92555 

-3.45 

0.0136 



Effect Test 




Source 

Nparm 

DF Sum of Squares 

F Ratio 

Prob>F 

Aircraftl 

1 

1 

72019961 

65.8970 

0.0002 

Aircraft2 

1 

1 

1785068 

1.6333 

0.2485 

Aircraft3 

1 

1 

25482135 

23.3157 

0.0029 

Aircraft2*Aircraft3 

1 

1 

13007898 

11.9020 

0.0136 
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Analysis of Variance 


Source 

DF 

Sum of Squares 

Mean Square 

F Ratio 

Model 

4 

127597945 

31899486 

29.1875 

Error 

6 

6557500 

1092917 

Prob>F 

C Total 

10 

134155444 


0.0005 


1000 - 

500 - 

0 “ ■ 


-500 - 

-1000 - 

-1500 - 

-10000 


“I-'-1- 1 -1- 1 -1- 1 —I- 1 -T" 

0 5000 10000 20000 

weighted objective Predicted 


Residual weighted object 



- 3 - 2-10123 


Normal Quantile 



Quantiles 


maximum 

100.0% 

1048.8 


99.5% 

1048.8 


97.5% 

1048.8 


90.0% 

1008.0 

quartile 

75.0% 

591.2 

median 

50.0% 

173.0 

quartile 

25.0% 

-691.7 


10.0% 

-1459.2 


2.5% 

-1559.7 


0.5% 

-1559.7 
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minimum 0.0% 


-1559.7 


Moments 

Mean 

Std Dev 

Std Error Mean 

Upper 95% Mean 

Lower 95% Mean 

N 

Sum Weights 


0.0000 
809.7839 
244.1590 
544.0237 
-544.0237 
11.0000 
11.0000 


Test for Normality 
Shapiro-Wilk WTest 

W Prob<W 
0.940458 0.5044 
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Appendix 6: Objective Value Average By Aircraft Type 


Number of 


rcraft 

A1 

0 

- 32.11742424 

1 

- 620.5530303 

2 

- 62.88571429 

3 

38.72727273 

4 

- 371.6621622 

5 

- 220.2356322 

6 

-127.3133333 

7 

191.5944444 

8 

-273.3733333 

9 

- 549.3431373 

10 

388.7971014 

11 

948.7111111 

12 

-2512 

13 

- 290.2708333 

14 

- 367.6212121 

15 

142.1833333 

16 

- 660.6060606 

17 

- 34.375 

18 

- 368.3181818 

19 

- 1475.65 

20 

- 362.1041667 

21 

- 521.9761905 

22 

- 2375.384615 

23 

- 3204.083333 

24 

- 1023.583333 

25 

- 2862.766667 

26 

- 1796.962963 

27 

- 925.7 

28 

- 5230.958333 

29 

- 3203.066667 

30 

- 1238.916667 

31 

- 342.9166667 

32 

- 1996.722222 

33 

- 3238.333333 

34 

- 3602.277778 

35 

- 4036.611111 

36 


37 

- 1205.166667 

38 

- 647.0833333 

39 

- 6823.333333 

40 

- 4604.388889 

41 

- 4676.5 

42 

-2935 

43 

- 7308.666667 

44 

- 815.1666667 


Type of Aircraft 
A2 A3 

595.6432 127.8956386 
805.3023 974.8809524 
1092.8 -121.4733333 
- 457.8 174.9416667 

205.9097 - 97.58928571 
508.4386 - 752.6018519 
- 367.765 - 130.5238095 
- 1414.72 619.1875 

- 156.167 - 544.7651515 
- 192.833 - 581.8181818 
365.625 - 358.8452381 
- 1055.29 - 528.1666667 
492.8333 - 1360.833333 
- 1944.92 - 132.8333333 
- 992.667 134.5606061 
- 1941.6 - 155.5333333 
591.0714 - 176.0625 

- 1178.35 - 4926.619048 
- 2566.22 - 2837.916667 
- 1143.3 - 2002.333333 
- 926.375 - 2286.125 

98.84848 - 1198.515152 
- 536.806 - 2586.666667 
- 1941.14 - 1707.5 

651.1458 1171.458333 
- 2405.61 - 601.6666667 
- 933.796 - 1600.333333 
- 2525.25 - 1116.638889 
- 1414.03 505.1666667 
1895.083 2321.25 

118.75 330.25 

- 374.867 - 2661.916667 
- 2762.42 - 3557.681818 
- 9002.83 - 4805.733333 
- 2706.07 - 4395.777778 
- 4316.5 - 659.75 

-3928 - 4169.055556 

- 1891.03 - 1889.722222 
- 5010.08 - 3282.212121 
-7969 - 5796.694444 

- 2135.72 - 4601.444444 
- 5187.57 - 3863.233333 
- 3216.63 - 3295.055556 
- 3568.87 - 4318.75 

- 3785.83 - 1888.145833 


A4 A5 
- 602.596 212.3323 
- 2176.33 531.1296 
- 1326.79 -245.548 
- 374.471 32.17778 
- 775.897 735.25 
1073.56 - 2034.32 
52.4023 - 1804.62 
557.0625 -4146 

629.7105 - 2462.23 
- 520.005 - 4707.53 
570.1739 - 3346.88 
- 1033.01 - 3167.42 
793.75 - 3171.47 
- 1038.55 - 3821.13 
- 1732.01 - 5578.44 
- 775.083 4414.556 
- 1417.89 - 2543.08 
- 911.716 - 255.444 
1507.808 - 1231.88 
- 2136.88 - 1681.88 
317.9444 - 2302.28 
- 1841.1 - 1789.98 
- 5.52083 - 3593.89 
- 1512.31 303.7667 
1237.389 - 8055.17 
1266.028 - 2434.58 
- 388.296 - 2577.2 
38.95833 - 1213.08 
- 120.5 - 1670.75 

- 663.7 - 1824.08 

- 1857.87 - 2319.57 
- 200.667 - 767.3 

- 1366.32 - 3335.61 
- 2859.12 - 4110.5 
- 2294.19 - 665.083 
- 4154.56 - 3836.5 
- 3501.11 - 4142.14 
- 1069.38 - 1534.9 
- 6068.58 -2916 

- 3970.96 - 1544.03 
- 10639.2 - 1822.88 
- 4550.21 - 1465.23 
- 2047.75 - 3862.62 
- 2147.83 - 2997.83 
- 2037.88 - 2922.33 
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45 

46 

47 

48 

49 

50 

51 

52 

53 

54 

55 

56 

57 

58 

59 

60 
61 
62 
63 


- 4054.142857 - 2117.9 
- 7486.25 - 1898.42 

- 5013.166667 - 5517.73 
- 5393.708333 - 1734.5 
- 3389.458333 -4776 

- 6167.166667 - 879.722 
-6872 - 5338.78 

- 2559.333333 - 4167.94 
- 1942.722222 
- 5585.666667 - 3605.67 
- 4310.166667 - 6998.78 
- 6649.8 - 4331.17 

- 5370.5 - 6390.78 

- 1350.3 - 5118.03 

- 3384.642857 - 5344.86 
- 4998.619048 - 4625.34 
- 4428.407407 - 7627.07 
- 4598.6875 - 3351.97 

- 6328.890351 - 5316.25 


- 3016.625 

- 2865.888889 

- 3967.018519 

- 1241.25 

- 3609.833333 

-2071 

- 2503.375 

- 35.58333333 

- 3330.979167 

- 7353.083333 

-4767 

- 4501.75 

- 11622.33333 

- 4045.583333 

- 4101.541667 

- 3541.796296 

- 5290.095238 

- 4761.8 

- 6364.425 


- 1450.26 - 5941.89 
- 2328.92 - 4683.83 
- 7384.77 770.9444 
- 5382.46 - 4072.17 
- 9036.92 - 1230.67 
- 5099.92 512.0833 
- 6417.5 -5363 

- 4928.42 - 2233.98 
- 3727.86 - 1586.08 
- 8630.67 394.5 

- 1197.25 
- 4934.38 - 3704.97 
- 2048.83 
- 2055.83 
- 8532.67 - 718.63 
- 1686.29 
- 4865.67 - 2695.17 
- 7915.6 

- 4983.59 - 4036.31 
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Appendix 7: Bennett’s Controller Interface 


@echo off 
rem *** 


rem *** 
rem *** 

File: master.bat 

rem ** 
rem ** 

Multi-Scenario batch File 

rem ** 
rem ** 

Written by: Barry D. Bennett Jr. 

rem ** 
rem ** 
rem ** 

Language: DOS 


rem ** Change aircraft numbers in datacft module of gams_in.txt 
call modify 

rcm ********************************* 

rem * * setup scenario input files and run scenario 1 

rcm 

copy msndatl.txt msndat.txt 
copy profilel.txt profiles.txt 
copy profwxl .txt profwx.txt 
rem ** 

rem ** Call scenario 1 batch file 

rem ** 

call tl_batch 

rem ** output to -aOO 

rem *** setup scenario input files and run scenario 2 

I*Cm ********************************* 

copy msndat2.txt msndat.txt 

copy profile2.txt profiles.txt 

copy profwx2.txt profwx.txt 

rem ** 

rem ** 

rem ** Call scenario 2 batch file 

rem ** 

call t2_batch 

rem ** output to -aOl 

rcm *****************************^*^^ 
rem *** setup scenario input files and run scenario 3 

r C m ^ ^ ^ 5jc jjc jjC * ^ ^ ^ 5|c?[c^C5jc^C5jC5jc^C5jC5jC5|C3 | Jc^'C5j< 

copy msndat3.txt msndat.txt 
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copy profile3.txt profiles.txt 
copy profwx3.txt profsvx.txt 
rem ** 
rem ** 

rem ** Call scenario 3 batch file 

rem ** 

call t3_batch 

rem ** output to -a02 

5jc >$C jjC5jC5)C5jC5jC5|C5!c5jC3|C jjc 2$C jj< 5|C 5|C l|c 5jC 5|c 5|c jjc j|c ?jc 

rem *** Collect run data and calculate value function 

rem ^ ^ ^ ^ ^ * ^Jc #Jc jjc ^ ^Jc 5jc jjc * jjc 5jc 3|C #jc Sjc 5$C 5fc jjc ^Jc ^fc 3^C ijc 

call objvalue 
rem ** 

rem** *END OF Multi-Scenario BATCH RUN 
exit 
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rem ********************************************* 
rem*** File: tl_batch.bat 

rem *** 

rem ** Scenario 1 batch File 

rem ** 

rem ** Written by: Barry D. Bennett Jr. 

rem ** 

rem ** Language: DOS 

rem ** 

rem ********************************************* 
rem ** 

rem ** Add new force structure info to gams input file 
rem ** 
rem ** 

copy oneA.txt+newslug.txt+oneB.txt gams_in.txt 
rem ** 
rem ** 

copy budgOOl .txt budg00.txt 
copy budg01_l .txt budgOl .txt 
copy budg02_l.txt budg02.txt 
rem ** 

cd\cfam99~ 1 \runfiles 

rem **************************************** 
rem ** Conduct the CFAM run 
rem ************************************* 
gams qstrikeg 
rem ** 

rem ************************************* 
rem ** Save the CFAM run output files 

copy c :\cfam99~ 1 \outfiles\acc2. csv c :\cfam99~ 1 \outfiles\acc2-a00 .csv 
copy c:\cfam99~l\outfiles\form2.csv c:\cfam99~l\outfiles\form2-a01 .csv 
copy c:\cfam99~l\outfiles\solvar2.csv c:\cfam99~l\outfIles\solvar2a.csv 
copy c:\cfam99~l \outfiles\sum2.csv c:\cfam99~ 1 \outfiles\sum2-a00.csv 
copy c:\cfam99~l\outfiles\time2.csv c:\cfam99~l\outfiles\time2-a0.csv 
rem ** 
rem ** 

cd\cfam99~ 1 \infiles 

rem ** *END OF BATCH RUN ******************************************* 
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********************************************* 

1 will 

rem *** File: t2_batch.bat 

rem *** 

rem ** Scenario 2 batch File 

rem ** 

rem ** Written by: Barry D. Bennett Jr. 

rem ** 

rem ** Language: DOS 

rem ** 

rem ********************************************* 
rem ** 

rem ** Add new force structure info to gams input file 
rem ** 

copy twoA.txt+newslug.txt+twoB.txt gams_in.txt 
rem ** 
rem ** 

copy budg00_2.txt budg00.txt 
copy budg01_2.txt budg01.txt 
copy budg02_2.txt budg02.txt 
rem ** 

cd\cfam99~ 1 \runfiles 

rem **************************************** 
rem ** Conduct the CFAM run 
rem ************************************* 
gams qstrikeg 
rem ** 

rem ************************************* 
rem ** Save the CFAM run output files 

r^m ************************************* 

rem 

copy c:\cfam99~l\outfiles\acc2.csv c:\cfam99~l\outfiles\acc2-a01.csv 
copy c:\cfam99~l\outfiles\form2.csv c:\cfam99~l\outfiles\form2-al.csv 
copy c:\cfam99~l\outfiles\solvar2.csv c:\cfam99~l\outfiles\solvar2b.csv 
copy c:\cfam99~l\outfiles\sum2.csv c:\cfam99~l\outfiles\sum2-a01 .csv 
copy c:\cfam99~l\outfiles\time2.csv c:\cfam99~l\outfiles\time2-al.csv 
rem ** 
rem ** 

cd\cfam99~l \infiles 

rem ** *END OF BATCH RUN ******************************************* 
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rem *** File: t3_batch.bat 

rem *** 

rem ** Scenario 3 batch File 

rem ** 

rem ** Written by: Barry D. Bennett Jr. 

rem ** 

rem ** Language: DOS 

rem ** 

rcm ********************************************* 
rem ** 
rem ** 

rem ** Add new force structure info to gams input file 
rem ** 
c: 

cd\cfam99~ 1 \infiles 

copy threeA.txt+newslug.txt+threeB.txt gams_in.txt 
rem ** 
rem ** 

copy budg00_3.txt budg00.txt 
copy budg01_3.txt budg01.txt 
copy budg02_3.txt budg02.txt 
cd\cfam99~ 1 \runfiles 

rem ^Jc ?Jc ?jc 5|C $jc * 5jc ** * * * 5^C 5>)c jjc ?|c 5|c ?jc 5jc 

rem ** Conduct the CFAM run 
rcm ************************************* 
gams qstrikeg 
rem ** 

rcm ************************************* 
rem ** Save the CFAM run output files 

copy c:\cfam99~l\outfiles\acc2.csv c:\cfam99~l\outfiles\acc2-a02.csv 
copy c:\cfam99~ 1 \outfiles\form2.csv c:\cfam99~ 1 \outfiles\form2-a02.csv 
copy c:\cfam99~l\outfiles\solvar2.csv c:\cfam99~l\outfiles\solvar2c.csv 
copy c:\cfam99~l\outfiles\sum2.csv c:\cfam99~l\outfiles\sum2-a02.csv 
copy c :\cfam99~ 1 \outfiles\time2 .csv c:\cfam99~l \outfiles\time2-a2 .csv 
rem ** 
rem ** 

cd\cfam99~ 1 \infiles 

rcm ** *END OF BATCH RUN ******************************************* 


103 



‘ *** File: modify.exe 
6 *** 

‘ ** GAMS input file modification routine. 

* ** 

‘ ** This program reads in the new candidate force structures provided by the 
‘ ** GENESIS GA and modifies the aircraft numbers in the GAMS input file for the 
‘ ** CFAM runs. CFAM requires that the minimum aircraft number in any aircraft 

‘ ** type be one. To insure this aircraft is not used, a sortie rate of zero is given to that 

‘ ** aircraft. 

‘ ** 

‘ ** Written by: Barry D. Bennett Jr. 

‘ ** 

‘ Language: Visual Basic 6.0 
‘ ** 

Private Sub Form_Load() 

Dim txt As String 
Dim Acft() As String 
Dim AcftType As String 

Dim Acft8 As String, Acft9 As String, AcftlO As String, Acftl8 As String 

Dim Acftl9 As String 

Dim A() As String 

Dim B() As String 

Dim I As Integer 

Dim J As Integer 

Dim X As Integer 

Open "C:\Cfam99~l\Infiles\oldslug.txt" For Input As #1 
Open "C:\Cfam99~l\Infiles\numacft.txt" For Input As #2 
Open "C:\Cfam99~l\Infiles\newslug.txt" For Output As #3 

Open "C:\Cfam99~l\Infiles\tempmod.txt" For Output As #4 
« ** 

‘ ** Read in the aircraft numbers from the new force structure 

‘ ** 

Line Input #2, AcftType 

Acft = Split(SuperTrim(AcftType)) 

For k = LBound(Acft) To UBound(Acft) 

If k = 0 Then Acft8 = Acft(k) 

If k = 1 Then Acft9 = Acft(k) 

If k = 2 Then AcftlO = Acft(k) 

If k = 3 Then Acftl8 = Acft(k) 

If k = 4 Then Acftl9 = Acft(k) 

Next k 

Print #4, Acft8, Acft9, AcftlO, Acft 18, Acft 19 
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c *** 

‘ * * Read in aircraft number data table from current section of the GAMS input file. 

‘ ** Modify it to reflect change in aircraft numbers and write the changed lines into a 

‘ ** new file. 


Do Until EOF(l) 

Line Input #1, txt 
A = Split(SuperTrim(txt)) 

B = Split(A(LBound(A)),".") 

Print #4, txt 

For k = LBound(A) To UBound(A) 

Print #4, "k= ", k, ”A(k)=", A(k), "B(k)=", B(k) 

Next k 

IfB(2) = "Tl" Then 

If B(l) = "8" Then 
If Acft8 = "0" Then 

Print #3, A(0); Tab(19); "1"; Tab(29); "0.0000" 
Else 

Print #3, A(0); Tab(19); Acft8; Tab(29); A(2) 
End If 

Elself B(l) = "9" Then 
If Acft9 = "0" Then 

Print #3, A(0); Tab(19); "1"; Tab(29); "0.0000" 
Else 

Print #3, A(0); Tab(19); Acft9; Tab(29); A(2) 
End If 

Elself B(l) = "10" Then 
If AcftlO = "0" Then 

Print #3, A(0); Tab(19); "1"; Tab(29); "0.0000" 
Else 

Print #3, A(0); Tab(19); AcftlO; Tab(29); A(2) 
End If 

Elself B(l) = "18" Then 
If Acftl8 = "0" Then 

Print #3, A(0); Tab(19); "1"; Tab(29); "0.0000" 
Else 

Print #3, A(0); Tab(19); Acftl8; Tab(29); A(2) 
End If 

Elself B(l) = "19" Then 
If Acftl9 = "0" Then 

Print #3, A(0); Tab(19); "1"; Tab(29); "0.0000" 
Else 

Print #3, A(0); Tab(19); Acftl9; Tab(29); A(2) 
End If 

Else 
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Print #3, A(0); Tab(19); A(l); Tab(29); "0.0000" 

End If 
Else 

Print #3, A(0); Tab(19); A(l); Tab(29); A(2) 

End If 

Loop 
Close #1 
Close #2 
Close #3 
Close #4 
End 

End Sub 

c ************************************* 

‘ ** 

‘ ** Subroutine to reduce the number of blank spaces between words on a line of text to 

‘ ** one. 

<■ ** 

t ************************************* 

Public Function SuperTrim(TheString As String) As String 
Dim temp As String, DoubleSpaces As String 
DoubleSpaces = Chr(32) & Chr(32) 
temp = Trim(TheString) 
temp = Replace(temp, DoubleSpaces, Chr(32)) 

Do Until InStr(temp, DoubleSpaces) = 0 

temp = Replace(temp, DoubleSpaces, Chr(32)) 

Loop 

SuperTrim = temp 

End Function 

t *********** g n£ j ofModify.exe program ’i'********************* 
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‘ *** File: objvalue.exe 

4 *** 

‘ ** Force structure objective value evaluation routine. 

‘ ** 

‘ ** This program reads in the results of the three CFAM scenario runs and 

‘ ** calculates 1) the objective value for each scenario, and 2) the objective value 

‘ ** for the multi-scenario. It outputs the multi-scenario evaluation to a file called 

‘ ** evalue.txt for use by the GENESIS GA. It also outputs a summary of the 

‘ ** structure evaluation to a file called report.txt. This one line summary includes the 

‘ ** evaluated force structure, the three scenario objective function values, and the 

‘ ** total multi-scenario objective function value. 

‘ ** 

‘ ** Written by: Barry D. Bennett Jr. 

‘ ** 

‘ ** Language: Visual Basic 6.0 
‘ ** 

Private Sub Form_Load() 

Dim txt As String 
Dim Acft() As String 
Dim AcftType As String 
Dim temp() As String 
Dim DO 5) As Double 
Dim CO As String 
Dim AO As String 
Dim BO As String 

Dim O As Double, 01 As Double, 02 As Double, 03 As Double 

Dim I As Integer, J As Integer, L As Integer, LM As Integer, Atotal As Integer 

Dim C1 As Integer, C2 As Integer, C3 As Integer 

Dim Acft8 As Integer, Acft9 As Integer, AcftlO As Integer, Acftl8 As Integer 
Dim Acftl9 As Integer 

Open "C:\Cfam99~l\Outfiles\acc2-aOO.csv" For Input As #1 
Open "C:\Cfam99~l\Outfiles\acc2-a01.csv" For Input As #2 
Open "C:\Cfam99~l\Outfiles\acc2-a02.csv" For Input As #3 
Open "C:\Cfam99~l\Outfiles\Object.txt" For Output As #4 
Open "C:\Cfam99~l\Infiles\tempobj.txt" For Output As #8 
Open "C:\Cfam99~l\Infiles\numacft.txt" For Input As #6 
Open "C:\Cfam99~l\Infiles\evalue.txt" For Output As #7 
Atotal = 0 
I = 0 
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»******************************** 

6 ** 

‘ ** Calculate total number of Aircraft in Force Structure 

4 ** 

6 ******************************** 

Line Input # 6 , AcftType 

Acft = Split(SuperTrim(AcftType)) 

For k = LBound(Acft) To UBound(Acft) 

If k = 0 Then Acft 8 = Acft(k) 

If k = 1 Then Acft9 = Acft(k) 

If k = 2 Then AcftlO = Acft(k) 

If k - 3 Then Acftl 8 - Acft(k) 

If k = 4 Then Acftl 9 = Acft(k) 

Nextk 

Atotal = Acft 8 + Acft9 + AcftlO + Acftl 8 + Acft 19 
Print #4, "Total number of Aircraft =", Atotal 

c ************************************* 
t ** 

‘ ** Calculate Scenario 1 Objective Function Parameters 

c ** 

t ************************************* 


Do 

Line Input #1, txt 
A = Split(SuperTrim(txt), 

J = LBound(A) 

L = UBound(A) 

If UBound(A) o -1 Then 

For k = LBound(A) To UBound(A) 
If A(k) = "1.00" Then 
D(0) = A(2) 

Cl =InStr(l, A(3), "YES") 

If Cl oOThen 
D(l) = 1 
Else 

D(1) = 0 
End If 
D(2) = A(4) 

D(3) = A(6) 

D(4) = A(8) 

End If 
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Next k 
End If 

Loop Until EOF(l) 

« ** 

‘ ** Calculate Scenario 2 Objective Function Parameters 
< ** 


Do 

Line Input #2, txt 
A = Split(SuperTrim(txt),",") 

J = LBound(A) 

L = UBound(A) 

If UBound(A) o -1 Then 

For k = LBound(A) To UBound(A) 
If A(k) = " 1.00" Then 
D(5) = A(2) 

Cl =InStr(l,A(3), "YES") 

If Cl o0 Then 
D(6) = 1 
Else 

D(6) = 0 
End If 
D(7) = A(4) 

D(8) = A(6) 

D(9) = A(8) 

End If 


Next k 
End If 

Loop Until EOF(2) 
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6 ************************************* 

C ** 

‘ ** Calculate Scenario 3 Objective Function Parameters 
« ** 

c Hi************************************ 


Do 

Line Input #3, txt 
A = Split(SuperTrim(txt),",") 

J = LBound(A) 

L = UBound(A) 

If UBound(A) o -1 Then 

For k = LBound(A) To UBound(A) 

If A(k) = "1.00" Then 
D(10) = A(2) 

Cl =InStr(l, A(3), "YES") 

If Cl o0 Then 
D(11) = 1 
Else 

D(11) = 0 
End If 

D(12) = A(4) 

D(13) = A( 6 ) 

D(14) = A( 8 ) 

End If 

Next k 
End If 

Loop Until EOF(3) 

c ************************************* 
c ** 

‘ Calculate the three scenario objective values: 01,02,03 

6 ** 

6 ************************************* 


01 = (-50 * D(0)) + 10000 * D(l) + 125 * D(2) - 50 * D(3) - 5 * D(4) - 50 * (Atotal) 
02 = (-50 * D(5)) + 10000 * D(6) + 125 * D(7) - 50 * D(8) - 5 * D(9) - 50 * (Atotal) 
03 = (-50 * D(10)) + 10000 * D(11) + 125 * D(12) - 50 * D(13) - 5 * D(14) - 50 * 
(Atotal) 
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6 ** 

‘ ** Calculate the multi-scenario objective values: O 

‘ ** using equal probabilities of occurrence for each scenario. 

‘ ** 

0 = (01 / 3)+ (02/3)+ (03/3) 

Print #4, "Objective values = ", 01,02, 03, O 

‘ ** Append to file report.txt the analyzed force structure, objective values obtained in 
‘ ** each scenario, and the overall objective function value. 

‘ ** 

Open "C:\Cfam99~l\Infiles\report.txt" For Append As #5 

Print #5, Acft8; Tab(7); Acft9; Tab(12); AcftlO; Tab(17); Acftl8; Tab(22); Acftl9; 
Tab(27); 01; Tab(37); 02; Tab(47); 03; Tab(57); O 
Close #5 

Print #8, Acft8; Tab(7); Acft9; Tab(12); AcftlO; Tab(17); Acftl8; Tab(22); Acftl9; 
Tab(27); 01; Tab(37); 02; Tab(47); 03; Tab(57); O 

‘ ** 

‘ ** Write to file evalue.txt the multi-scenario objective function value used by 

‘ ** GENESIS GA. 

‘ ** 

Print #7, O 
Close #1 
Close #2 
Close #3 
Close #4 
Close #6 
Close #7 
Close #8 
End 

End Sub 
‘ ** 

‘ ** Subroutine to reduce the number of blank spaces between words on a line of text to 

‘ ** one. 

£ ** 

Public Function SuperTrim(TheString As String) As String 
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Dim temp As String, DoubleSpaces As String 

DoubleSpaces = Chr(32) & Chr(32) 

temp = Trim(TheString) 

temp = Replace(temp, DoubleSpaces, Chr(32)) 

Do Until InStr(temp, DoubleSpaces) = 0 
temp = Replace(temp, DoubleSpaces, Chr(32)) 

Loop 

SuperTrim = temp 

End Function 

t End of Objvalue cxc program 
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Appendix 8: Bennett’s CFAM Setui 
Campaign Phase Goals 


Goals 


Target Class 

Defeat Close Forces 


Defeat 2nd Echelon Forces 


Defeat Naval Forces 


Reduce Sortie Capacity 


Degrade Air Defenses 


Degrade Air Control 


Destroy Weapons of Mass Destruction 


Destroy Economic/Military Capacity 


Destroy Leadership Sites 


Destroy Lines of Communication 
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Campaign Phase Switch Settings 


Theater! Switch (Y/N) 

Class ID Target Class_Phasel Phase2 Phase3 Phase4 


1 

Defeat Close Forces 

Y 

Y 


Y 

2 

Defeat 2nd Echelon Forces 

Y 

Y 

Y 

Y 

3 

Defeat Naval Forces 


Y 

Y 

Y 

4 

Reduce Sortie Capacity 

Y 


Y 

Y 

5 

Degrade Air Defenses 

Y 

Y 

Y 

Y 

6 

Degrade Air Control 

Y 

Y 

Y 

Y 

7 

Destroy Weapons of Mass Destruction 




Y 

8 

Degrade Economic/Military Capacity 




Y 

9 

Destroy Leaderships Sites 



Y 

Y 

10 

Destroy Lines of Communication 

Y 

Y 

Y 

Y 


Theater2 Switch (Y/N) 

Class ID Target Class_Phasel Phase2 Phase3 Phase4 


1 

Defeat Close Forces 


Y 


Y 

2 

Defeat 2nd Echelon Forces 


Y 

Y 

Y 

3 

Defeat Naval Forces 


Y 

Y 

Y 

4 

Reduce Sortie Capacity 

Y 

Y 


Y 

5 

Degrade Air Defenses 

Y 

Y 

Y 

Y 

6 

Degrade Air Control 

Y 

Y 

Y 

Y 

7 

Destroy Weapons of Mass Destruction 


Y 


Y 

8 

Degrade Economic/Military Capacity 




Y 

9 

Destroy Leaderships Sites 



Y 

Y 

10 

Destroy Lines of Communication 

Y 

Y 

Y 

Y 


Theater3 Switch (Y/N) 

Class ID Target Class_Phasel Phase2 Phase3 Phase4 


1 

Defeat Close Forces 

Y 

Y 


Y 

2 

Defeat 2nd Echelon Forces 

Y 

Y 

Y 

Y 

3 

Defeat Naval Forces 


Y 

Y 

Y 

4 

Reduce Sortie Capacity 

Y 



Y 

5 

Degrade Air Defenses 

Y 

Y 

Y 

Y 

6 

Degrade Air Control 

Y 

Y 

Y 

Y 

7 

Destroy Weapons of Mass Destruction 




Y 

8 

Degrade Economic/Military Capacity 




Y 

9 

Destroy Leaderships Sites 



Y 

Y 

10 

Destroy Lines of Communication 

Y 

Y 

Y 

Y 
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Theater Targets by Type 


Target ID Elements 


1 

2 

3 

4 

5 

6 

7 

8 

9 

10 
11 
12 

13 

14 

15 

16 

17 

18 

19 

20 
21 
22 

23 

24 

25 

26 

27 

28 

29 

30 

31 

32 

33 

34 

35 

36 

37 

38 

39 

40 

41 

42 


1 

2 

12 

4 

1 

10 

10 

24 

16 

10 

6 

1 

1 

4 

1 

1 

30 

1 

1 

1 

1 

1 

8 

2 

1 

1 

1 

1 

14 

1 

1 

6 

12 

3 

6 

3 

1 

1 

3 

3 
1 

4 


Description 

Ferry 

Railroad Yard Choke Point 

Truck Column 

MLRS in Position 

Air to Surface Missile in Travel 

Open Storage, Inert Supplies 

Open Storage, Munitions 

Truck Park 

Bridge Type 1 

Bldg Storage 

Iron/Steel Plant (Blast Furnaces) 

Bldg w/Machine Tools 

Bldg. Manufacturing/ Assembly 

POL, Airfield POL Storage, small 

Bunker, Arch 

Bunker, Aircraft 

POL, Airfield POL Storage 

Runway-A/C Highway Strip 

Runway-Earth Hard Soil 

Runway-Asphalt 

Taxiway-Concrete/Asphalt 

Acft Bunker, Double Shell 

Acft Bunker, Large 

Acft Hanger, Steel 

Taxiway Bridge 

Bridge, Pontoon 

Bridge, Highway 

Bridge, Dual Purpose 

Port Facilities, Cranes 

Marine Facility Tunnel Adit 

Port Facility Transverser 

Transformer Yard Large 

Transformer Yard Small 

POL Pump Station 

POL Storage Tanks Above Ground 

POL Refineries Rupture Kill 

Ship, Carrier 

Ship, Cruiser 

Newer Tank Platoon 

Modern Tank Platoon 

Radar in Open, M-Kill 

SAM Missile System 


Number of Targets by Type 
Theater 1 Theater 2 Theater 3 


1 2 1 

3 1 2 

2 1 1 

2 0 2 

1 3 2 

1 2 1 

2 4 1 

1 0 1 

2 1 2 

3 2 1 

2 1 2 

6 3 4 

3 2 3 

1 3 2 

2 4 8 

6 18 12 

2 3 2 

3 6 2 

0 0 1 

2 4 3 

2 4 3 

10 15 8 

2 4 5 

9 20 11 

0 1 0 

0 1 2 

4 5 5 

3 5 4 

1 0 1 

1 0 2 

2 1 2 

2 1 2 

1 2 1 

2 1 2 

2 1 2 

1 2 1 

0 0 1 

3 0 3 

20 1 10 

5 1 3 

2 5 3 

2 5 5 
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Number of Targets by Type 


Target ID 

Elements 

Description 

Theater 1 

Theater 2 

Theater 3 

43 

1 

Radar, SAM site Msn Kill 

5 

10 

4 

44 

3 

SAM Telars 

2 

6 

6 

45 

1 

Bunker, Command & Control 

2 

2 

3 

46 

1 

Bunker, Generic Sector OPS Center 

1 

2 

3 

47 

2 

Radar, Revetted EW/GCI 

1 

4 

3 

48 

1 

Radar, EW/GCI Site 

1 

2 

3 

49 

28 

Aircraft, Fighter Newer 

1 

2 

3 

50 

30 

Aircraft, Bomber 

2 

1 

1 

51 

10 

Acft, Helicopter 

2 

1 

0 

52 

7 

Acft, Fighter Old 

3 

2 

1 

53 

420 

Personnel, Bivovac/Assembly 

1 

0 

1 

54 

200 

Personnel, Prone 

1 

0 

1 

55 

31 

Modern Tank Column 

1 

0 

1 

56 

9 

SAM, Short Range Fire Unit 

2 

4 

4 

57 

6 

ARTY, SPG in Position K-Kil 

3 

6 

4 

58 

6 

ARTY, XX-mm Twed Field Gun/How 

3 

2 

3 

59 

31 

APC Battalion (BMPs) 

1 

0 

1 

60 

3 

APC Platoon (BMPs) 

4 

2 

3 

61 

10 

APC Column (BMP) 

1 

0 

1 

62 

1 

Radar SAM Short 

4 

8 

6 

63 

1 

Radar, Sam Medium 

6 

12 

8 

64 

1 

SSM, Missile site, one missile in tel 

5 

11 

8 

65 

5 

Bldg. RIC/HSF with 10-25 T crane 

1 

2 

2 

66 

1 

Bldg. Nuclear Research Facility 

2 

1 

2 

67 

1 

Nuclear WPNS Support Bunker 

0 

0 

0 

68 

1 

Nuclear WPNS Support Bunker 

0 

0 

0 

69 

1 

Bunker, Command & Control 

3 

2 

2 

70 

1 

Bunker, Ministry of Defense 

1 

1 

1 



Total Target Entities 

1387 

727 

1413 


Scenario Target Distribution by Target Class 

Target Entities 


Class ID Target Class_ Theater 1 Theater 2 Theater 3 


1 

Defeat Close Forces 

313 

48 

280 

2 

Defeat 2nd Echelon Forces 

558 

24 

546 

3 

Defeat Naval Forces 

20 

1 

22 

4 

Reduce Sortie Capacity 

252 

336 

288 

5 

Degrade Air Defenses 

51 

111 

98 

6 

Degrade Air Control 

4 

12 

12 

7 

Destroy Weapons of Mass Destruction 

6 

14 

10 

8 

Degrade Economic/Military Capacity 

133 

147 

106 

9 

Destroy Leaderships Sites 

4 

3 

3 

10 

Destroy Lines of Communication 

46 

31 

48 


Total Target Entities 1387 727 1413 
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Theater Targets by Target Class 




Target 

Theater 1 

Theater 2 Theater 3 

Target Class: 

Defeat 2nd Echelon Forces 

Elements 

Targets 

Targets 

Targets 

Included Tgts: 

APC Battalion (BMPs) 

31 

1 

0 

1 


APC Column (BMPs) 

10 

1 

0 

1 


ARTY, xx-mm towed field gun/howitzer 

6 

3 

2 

3 


Modern Tank Column 

31 

1 

0 

1 


Personnel Bivouac/Assembly 

420 

1 

0 

1 


Truck Column 

12 

2 

1 

1 


Truck Park 

24 

1 

0 

1 


Total Target Entities 


558 

24 

546 



Target 

Theater 1 

Theater 2 Theater 3 

Target Class: 

Defeat Close Forces 

Elements 

Targets 

Targets 

Targets 

Included Tgts: 

APC Platoon (BMPs) 

3 

4 

2 

3 


ARTY, SPG in position, K-kill 

6 

3 

6 

4 


MLRS in Position 

4 

2 

0 

2 


Modern Tank Platoon 

3 

5 

1 

3 


Newer Tank Platoon 

3 

20 

1 

10 


Personnel Prone 

200 

1 

0 

1 


Total Target Entities 


313 

48 

280 




Target 

Theater 1 

Theater 2 Theater 3 

Target Class: 

Defeat Naval Forces 

Elements 

Targets 

Targets 

Targets 

Included Tgts: 

Marine Facility Tunnel ADIT 

1 

1 

0 

2 


Port Facilities Cranes 

14 

1 

0 

1 


Port Facility Transverser 

1 

2 

1 

2 


Ship, Carrier 

1 

0 

0 

1 


Ship, Crusier 

1 

3 

0 

3 


Total Target Entities 


20 

1 

22 



Target 

Theater 1 

Theater 2 Theater 3 

Target Class: 

Degrade Air Control 

Elements 

Targets 

Targets 

Targets 

Included Tgts: 

Bunker, Generic Sector Ops Center 

1 

1 

2 

3 


Radar, EW/GCI Site 

1 

1 

2 

3 


Radar, Revetted EW/GCI 

2 

1 

4 

3 


Total Target Entities 


4 

12 

12 
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Target Class: Degrade Air Defenses 

Target 

Elements 

Theater 1 
Targets 

Theater 2 Theater 3 
Targets Targets 

Included Tgts: Bunker, Command & Control 

1 

2 

2 

3 

Radar, SAM short 

1 

4 

8 

6 

Radar in open, M-kill 

1 

2 

5 

3 

Radar, SAM site MSN kill 

1 

5 

10 

4 

Radar, SAM medium 

1 

6 

12 

8 

SAM Missile System 

4 

2 

5 

5 

SAM Telars 

3 

2 

6 

6 

SAM, short range fire unit 

9 

2 

4 

4 

Total Target Entities 

Target Class: Degrade Economic/Military Capacity 

Target 

Elements 

51 

Theater 1 
Targets 

111 98 

Theater 2 Theater 3 
Targets Targets 

Included Tgts: Bldg, Storage 

10 

3 

2 

1 

Bldg, Manufacturing/Assembly 

1 

3 

2 

3 

Bldg, Nuclear Research Facility 

1 

2 

1 

2 

Bldg, RIC/HSF with 10-25 ton crane 

5 

1 

2 

2 

Bldg with machine tools 

1 

6 

3 

4 

Iron/Steel Plant (blast furnaces) 

6 

2 

1 

2 

Open Storage, Inert Supplies 

10 

1 

2 

1 

Open Storage, Munitions 

10 

2 

4 

1 

POL Pump Station 

3 

2 

1 

2 

POL Refineries, Rupture Kill 

3 

1 

2 

1 

POL Storage Tanks, above ground 

6 

2 

1 

2 

Transformer Yard, Large 

6 

2 

1 

2 

Transformer Yard, Small 

12 

1 

2 

1 

Total Target Entities 

Target Class: Destroy Leaderships Sites 

Target 

Elements 

133 

Theater 1 
Targets 

147 

Theater 2 
Targets 

106 

Theater 3 
Targets 

Included Tgts: Bunker, Command & Control 

1 

3 

2 

2 

Bunker, Ministry of Defense 

1 

1 

1 

1 

Total Target Entities 

Target Class: Destroy Lines of Communication 

Target 

Elements 

4 

Theater 1 
Targets 

3 3 

Theater 2 Theater 3 
Targets Targets 

Included Tgts: Ferry 

1 

1 

2 

1 

Bridge Type 1 

16 

2 

1 

2 

Bridge, Dual Purpose 

1 

3 

5 

4 

Bridge, Highway 

1 

4 

5 

5 

Bridge, Pontoon 

1 

0 

1 

2 

Railroad Yard Chokepoint 

2 

3 

1 

2 

Total Target Entities 


46 

31 

48 
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Target 

Theater 1 Theater 2 Theater 3 

Target Class: Destroy Weapons of Mass Destruction 

Elements 

Targets 

Targets 

Targets 

Included Tgts: Air to Surface Missile in travel mode 

1 

1 

3 

2 

Nuclear Wpns Support Bunker, earth covered 

1 

0 

0 

0 

Nuclear Weapons Support Bunker, earth covered 

1 

0 

0 

0 

SSM, Missile Site, one missile on TEL 

1 

5 

11 

8 

Total Target Entities 


6 

14 

10 



Target Theater 1 Theater 2 Theater 3 

Target Class: Reduce Sortie Capacity 

Elements Targets 

Targets 

Targets 

Included Tgts: Acft Hanger, Steel 

2 9 

20 

11 

Acft Bunker, Double Shell 

1 10 

15 

8 

Acft Bunker, Large 

8 2 

4 

5 

Aircraft, Helicopters 

10 2 

1 

0 

Aircraft, Bomber 

30 2 

1 

1 

Aircraft, Fighter Newer 

28 1 

2 

3 

Aircraft, Fighter Old 

7 3 

2 

1 

Bunker, Arch 

1 2 

4 

8 

Bunker, Aircraft 

1 6 

18 

12 

POL, Airfield POL Storage, small underground 

4 1 

3 

2 

POL, Airfield POL Storage 

30 2 

3 

2 

Runway - A/C Highway Strip 

1 3 

6 

2 

Runway - Asphalt 

1 2 

4 

3 

Runway - Earth, Hard Soil 

1 0 

0 

1 

Taxiway - Concrete/Asphalt 

1 2 

4 

3 

Taxiway Bridge 

1 0 

1 

0 

Total Target Entities 

252 

336 

288 
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Weapons Inventory 


ID 

Description 

Inventory 

43 

GBU-Rock Type 1 

50,000 

44 

GBU-Rock Type 2 

bh 

45 

GBU-Rock Type 3 


46 

GBU-Rock Type 4 

LESSEE 

11 

GBU-Rock Type 5 

50,000 

12 

GBU-Rock Type 6 

50,000 

13 

GBU-Rock Type 7 

50,000 

14 

GBU-Rock Type 8 

50,000 

15 

GBU-Rock Type 9 

WEMim 

16 

GBU-Rock Type 10 

M 1 171 ", 

17 

GBU-Rock Type 11 

50,000 

18 

GBU-Rock Type 12 


19 

GBU-Rock Type 13 

mm 

20 

GBU-Rock Type 14 


21 

GBU-Rock Type 15 


22 

GBU-Rock Type 16 

50,000 

23 

GBU-Rock Type 17 

50,000 

30 

GBU-Rock Type 18 

HB 

32 

GBU-Rock Type 19 

50,000 

36 

GBU-Rock Type 20 

50,000 

37 

GBU-Rock Type 21 

50,000 

38 

GBU-Rock Type 22 

50,000 

39 

GBU-Rock Type 23 

50,000 


ID 

Description 

Inventory 

25 

Smart Rock Type 1 

50,000 

26 

Smart Rock Type 2 

50,000 

27 

Smart Rock Type 3 

50,000 

28 

Smart Rock Type 4 

50,000 

29 

Smart Rock Type 5 

50,000 

31 

Smart Rock Type 6 

50,000 

33 

Smart Rock Type 7 

50,000 

34 

Smart Rock Type 8 

50,000 

35 


50,000 

40 



41 

Smart Rock Type 11 

50,000 

42 

Smart Rock Type 12 

mm 

1 

Rock Type 1 

■H 

2 

Rock Type 2 

50,000 

3 

Rock Type 3 

50,000 

4 

Rock Type 4 

50,000 

5 

Rock Type 5 

50,000 

6 

Rock Type 6 

50,000 

7 

Rock Type 7 

10,000 

8 

Rock Type 8 

50,000 

9 

Rock Type 9 

10,000 

10 

Rock Type 10 

50,000 

24 

Rock Type 11 

50,000 

47 

Powered Rock Type 1 

50,000 

48 

Powered Rock Type 2 

50,000 

49 

Powered Rock Type 3 

50,000 

50 

Powered Rock Type 4 

50,000 

51 

Powered Rock Type 5 

10,000 

52 

Powered Rock Type 6 

10,000 

53 

Powered Rock Type 7 

50,000 

54 

Powered Rock Type 8 

50,000 

55 

Powered Rock Type 9 

50,000 

56 

Powered Rock Type 10 

50,000 

57 

Powered Rock Type 11 

10,000 

58 

Powered Rock Type 12 

50,000 

59 

Powered Rock Type 13 

50,000 

60 

Powered Rock Type 14 

50,000 

61 

Powered Rock Type 15 

50,000 

62 

Powered Rock Type 16 

50,000 
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